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Executive Summary 

The present study was commissioned by the Massachusetts Department of Elementary and 
Secondary Education (ESE) to provide a profile of Massachusetts English Language Learners 
(ELLs) and policy recommendations for improving their outcomes. ESE was concerned about 
previous study findings that most of these learners exited ELL instructional programs at 
relatively fast rates—within three years of school entry as kindergartners—but struggled 
academically following exit. Furthermore, the State was concerned about the numbers of 
students who never exited ELL instructional services at all. The purpose of this report is to 
update and expand these findings to reflect changes in the standards and assessments for ELLs 
and a major statewide initiative to improve the training of core academic teachers of ELLs. 

We performed in-depth statistical analyses using district-level data for a sample of 10 select 
districts (“Phase 1” of the report) and State student demographic and assessment data (“Phase 2” 
of the report). In Phase 1, we looked at a rich set of academic indicators such as graduation rates 
as well as performance on statewide content assessments (the Massachusetts Comprehensive 
Assessment System [MCAS]) and the relevant statewide English language proficiency 
assessments (Massachusetts English Proficiency Assessment [MEPA] and ACCESS for ELLs). 
Nonacademic indicators examined include retention, dropout, chronic absenteeism, stability, and 
suspension rates. In Phase 2, we followed a kindergarten ELL cohort across most of their 
schooling trajectory (from kindergarten through 10th grade) in Massachusetts public schools to 
look at whether and when they were exited from ELL programs, when they became English 
proficient and how they fared on the MCAS both while still classified as ELL and after exit. In 
both phases of this study, we looked at variability among districts serving these learners. 

Phase 1 key findings include the following: 

■ Growth in ELL population: All sampled districts experienced substantial growth in 
their ELL population over the last decade, with the highest growth districts also among 
those with the highest concentrations of ELLs in the State. 

■ Distribution of ELLs: The average ELL student in Massachusetts attended “triply 
segregated” schools—those with high proportions of minority students, ELL students and 
students from low-income households—compared to the average non-ELL student. 

■ Clustering of students from common language groups: In six of the 10 districts, more 
than 50 percent of ELLs spoke a common language whereas in those same districts, the 
second most common language was spoken by fewer than 2 percent of ELLs in the 
district. 

■ Dual identification as ELL with disabilities: ELLs are typically identified for special 
education services at comparable rates as in the general student population. However, 
identification rates vary by district—we highlight a district in which ELLs are identified 
at rates higher than those in the overall population and one in which ELLs are identified 
at rates lower than those in the general population. 

■ Academic indicators of current ELLs: Overall performance on the State’s language 
proficiency assessment, ACCESS for ELLs, was low in many of the sampled districts, 
with 25 percent or fewer ELLs scoring proficient in nine of the 10 districts in 2013-14. 
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The percentage of ELLs perfonning at proficient or higher on the English language arts 
(ELA), mathematics, and science Massachusetts Comprehensive Assessment System 
(MCAS) in all 10 districts fell below the State average for all students in 2013-14. 

■ Nonacademic indicators: ELLs in most of the sampled districts had higher grade 
retention, dropout, chronic absenteeism, and suspension rates than the State averages. 
ELLs in most sampled districts have lower stability and four-year graduation rates than 
the State averages. 

Phase 2 key findings include the following: 

■ Time to English language proficiency: The average ELL in the sample takes 3.3 years 
after kindergarten entry to perform at the proficient level on the statewide English 
language proficiency assessment. ELL students who also receive special education 
'services and Spanish-speaking ELLs take longer to become proficient. 

■ Time to reclassification: The average ELL in the sample is reclassified after 2.7 years in 
Massachusetts schools. However, by eighth grade, more than 30 percent of former ELLs 
were not proficient in English language arts, and more than 60 percent were not 
proficient in mathematics. 

■ Long-term ELLs: Statewide, approximately 12 percent of students who began as ELLs 
in kindergarten did not achieve English proficiency by the end of the study period. The 
majority of these students were also identified as needing special education services, 
while only 7.3 percent of the initial samples were classified as needing special education 
services. 

Educating ELLs to college- and career-ready standards poses a challenge to Massachusetts, 
partially due to the rapid growth of this population in many districts across the State. In addition, 
this study shows that many long-term ELLs require specialized support because they are dual- 
identified as needing both special education and English language instructional support services. 
Moreover, ELLs tend to be clustered in the highest need districts in the State (including two 
Level 5 districts) with large proportions of other ELLs as well as minority students and students 
from low-income households who are also some of the State’s highest need learners. 

Large numbers of former ELLs also merit attention: many ELLs are outperformed by 
mainstream peers statewide (i.e., “never-ELL”) after exit: only 55 and 46 percent score proficient 
or above in ELA and mathematics, respectively, in fifth grade (compared to 66 and 56 percent of 
never-ELLs), and 68 and 39 percent are proficient in eighth-grade ELA and mathematics 
(compared to 83 and 55 percent of never-ELLs). Addressing the academic and nonacademic 
needs of ELLs across their entire schooling trajectory including after exit is critical to ensuring 
that these learners are provided with their federally guaranteed right to “participate 
meaningfully” in public school education programs (Office for Civil Rights [OCR], 2015). 

Nonetheless, Massachusetts ELLs should be considered a resource to the State. They bring a 
wealth of linguistic and cultural diversity to the State’s districts. Academic proficiency in more 
than one language is no longer simply an asset but a requirement to be competitive in our 
increasingly global workplace and world. At the end of the report, we provide detailed 
recommendations to promote ELLs’ outcomes. 
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Introduction 


English language learners (ELLs) 1 —students who speak a non-English language at home and 
who have not acquired sufficient academic English to perfonn ordinary classroom work in 
English—represent a substantial and growing population of students in Massachusetts. Over the 
10-year period from 2003-04 to 2013-14, K-12 ELL enrollment increased nearly 54 percent, 
from 49,297 students to 75,947 students; as a result, ELLs made up 7.8 percent of the State’s 
total student body in the 2013-14 school year. 2 3 This trend is not unusual. U.S. Grade PK-12 
ELL enrollment grew nationwide by more than 51 percent between the 1998-99 school year and 
the 2008-09 school year, while the growth of total student enrollment increased by only 7 
percent (National Clearinghouse for English Language Acquisition and Language Instruction 
Educational Programs [NCELA], 2011). ELLs are clustered in the state’s most high need urban 
districts, several of which have been identified as Level 4 and 5 districts—defined by the State as 
struggling academically based on an analysis of four-year trends in absolute achievement, 
student growth, and improvement trends as measured by MCAS. A recent Massachusetts study 
(Slama, 2014) found that 78 percent of the students who were identified as ELL at kindergarten 
entry in 2002 attended school in an urban district. 

Studies by the MBESE (2009), Owens (2010), and Slama (2014) have found that Massachusetts 
reflects another national ELL trend: ELLs in the United States tend to fare worse academically 
(NCES, 2010, 2011) and in other achievement areas, such as graduation rates (Rumberger, 

2006), than their non-ELL peers. Owens (2010) found that ELLs in Massachusetts were more 
likely than English-proficient students to repeat a grade and were 25 percent more likely to be 
suspended. The MBESE (2009) reported that although ELLs in Massachusetts fare better than 
other ELLs nationwide, they have a larger achievement gap and graduation rate gap with their 
non-ELL peers than in most other states. The achievement gap is not limited to students who 
remain classified as ELLs; Slama (2014) found that large percentages of ELLs struggled to keep 
up with mainstream classwork even several years after reclassification as English proficient into 
those classrooms. By fifth grade, fewer than half of the kindergarten ELL cohort who had been 
reclassified scored at the proficient or above level on the ELA and mathematics Massachusetts 
Comprehensive Assessment System (MCAS). 

The present study, sponsored by the Massachusetts Department of Elementary and Secondary 
Education (ESE), capitalizes on Massachusetts’ rich district- and student-level data sets to 
provide an updated profile of Massachusetts ELLs and recommendations for improving their 
outcomes. We begin in Phase 1 of the study with a description of ELL demographic trends and 
instructional and reclassification practices in 10 selectively sampled Massachusetts districts. 
Then, in Phase 2, we expand on Slama’s (2014) longitudinal study of Massachusetts ELL 
achievement, reproducing her analysis with a cohort of students that began kindergarten in 2003- 

1 English language learners are defined in Massachusetts as students who speak a non-English language at home, are 
less than proficient on the ELP assessment, and are unable to perform ordinary classwork in English. Retrieved from 
http://www.doe.mass.edu/infoservices/data/sims/SIMS-DataHandbook.pdf 

2 Retrieved from http://profiles.doe.mass.edu/state report/selectedpopulations.aspx 

3 Retrieved from http://www.mass.gov/edu/government/departments-and- 

boards/ese/programs/accountabilitv/support-for-level-3-4-and-5-districts-and-schools/school-and-district- 

turnaround/ 
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04 and following them through Grade 10. 4 The selection of the 2003-04 cohort of ELLs is 
significant because it corresponds with the statewide shift to English-only programs following 
the 2002 passage of Question 2, a ballot initiative that made Massachusetts one of only three 
states that mandate English-only instruction. 5 

Study Questions 

The study is organized into two phases. The first phase provides an aggregate district-level 
analysis of ELL student demographic, geographic, and academic program profiles across a 
purposive sample of 10 Massachusetts districts, with a focus on documenting key trends. The 
second phase is an in-depth investigation of a kindergarten cohort of ELLs, drawing on statewide 
data, to examine students’ academic trajectories over time from initial identification as ELL until 
10th grade when they may or may not have been reclassified as fluent English proficient. 
Research questions and key findings are listed in the following sections. 

In this section we include an overview of our research questions (RQs) by study phase. Key 
findings are highlighted in text boxes in the body of the report. 

Research Questions 

Phase 1: ELL Student Demographic, Geographic, and Academic Program Profile 

■ RQla. What are the key demographic characteristics of Massachusetts ELLs in the 10 
selected districts? 

» RQlb. What is the distribution of ELLs across schools within these 10 districts? 

■ RQlc. What are the key features of ELLs’ language proficiency programs, academic 
profiles (e.g., MCAS performance; retention, dropout, and graduation rates), and 
nonacademic profiles (e.g., chronic absenteeism, stability, and in-school and out-of¬ 
school suspension rates) in these 10 districts? How do these features vary by district? 

■ RQld, What are district guidelines for making reclassification decisions in a purposively 
selected sample of 10 districts? 

Phase 2: ELL Longitudinal Student Outcomes Analysis 

■ RQ2a. After initial classification as ELL in kindergarten in 2003, what is the average 
time to first reaching the following key academic milestones: 

1. Scoring proficient on the statewide English language proficiency assessments (MEPA 
or ACCESS) 

2. Being reclassified as former ELL 

■ RQ2b. What proportion of the 2003 kindergarten ELL cohort who scored proficient on 
the statewide English language proficiency assessments (MEPA or ACCESS) is 
reclassified as English proficient in the following year? 


4 Slama’s (2014) study examined students who began school in 2002-03 and followed them until Grade 7. 

5 California and Arizona are the other states with English-only instruction policies. 
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■ RQ2c. What proportion of the 2003 kindergarten ELL cohort scored proficient on 
statewide mathematics and English language arts content-area assessments (MCAS) 
while classified as ELL? What proportion scored proficient in mathematics and English 
language arts after first reclassification as former ELL? 

■ RQ2d. What is the relationship between the 2003 kindergarten ELL cohort’s 
performance on the statewide English language proficiency assessments (MEPA or 
ACCESS) and the cohort’s performance in mathematics and English language arts 
content-area assessments (MCAS)? 

■ RQ2e. What proportion of the 2003 kindergarten ELL cohort experienced on-time 
promotion by fifth and eighth grade? 

- RQ2f. How does the average time to reaching the key educational milestones defined in 
RQ2a vary across 10 purposively selected districts? 

We discuss these findings in greater detail in subsequent sections. First, in order to contextualize 
the study findings, we provide a brief overview of the ELL policy context in Massachusetts as 
situated within the larger national ELL policy landscape. Next, we provide a description of the 
research design and methods, followed by findings from each phase of the research. We then 
provide recommendations based on our findings and conclude with limitations of the current 
study and directions for future research. 

ELL Policy Context 

Federal statute guarantees ELLs the right to “participate meaningfully” in public school 
education programs, and both state and local education agencies that receive federal funding are 
obligated to act to help ELLs overcome language barriers that might interfere with such 
participation (see ASPIRA Consent Decree, 1974; Castaneda v. Pickard, 1981; Equal Education 
Opportunities Act; Lau v. Nichols, 1974). Recently, the U.S. Department of Justice (2015) 
released legal guidance to remind education agencies of this obligation. Practically this statutory 
obligation means providing language support in addition to content area instruction, although 
states have latitude in how they provide such support. 

In Massachusetts, Chapter 386 of the Acts of 2002, legislated in response to a public referendum 
popularly known as Question 2, mandates that instruction for ELLs be provided primarily in 
English. According to Chapter 71A of the Massachusetts General Laws (G.L. c. 71 A), all 
students classified as ELLs must be educated in a sheltered English immersion (SEI) program, 
unless a program waiver is sought for another program model (dual language instruction, 
however, is allowed). This requirement applies to all districts that enroll ELL students, regardless 
of the number of students. 

In Massachusetts, programs consist of two components: sheltered content instruction and English 
as a second language (ESL) instruction. Sheltered content instruction is defined as “approaches, 
strategies and methodology to make the content of lessons more comprehensible and to promote 
the development of academic language needed to successfully master content standards” (ESE, 
2013, pp. 12-13). Sheltered content instruction must be taught by qualified teachers and be 
aligned to the Massachusetts Curriculum Framework and World-Class Instructional Design and 
Assessment Consortium English language development standards. ESL instruction is defined as 
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that which “provides explicit, direct, and systematic instruction to learn the English language that 
is intended to promote second language acquisition and English language proficiency” (ESE, 
2013, p. 13). ESL instruction includes instruction that is tailored specifically to ELLs’ varying 
proficiency levels. 

This study indirectly examines these instructional policies by providing a longitudinal analysis of 
ELL outcomes in the cohort of students that began school in 2003, the year after the passage of 
Question 2. Specifically, it examines whether these practices “succeed, after a legitimate trial, in 
producing results indicating that students’ language barriers are actually being overcome within a 
reasonable period of time” (Office for Civil Rights [OCR], 2015, p. 6). The study also examines 
more thoroughly the policies, practices, and other factors contributing to students’ outcomes in 
10 purposively selected Massachusetts districts. The findings are analyzed and recommendations 
made with reference to recent Office for Civil Rights (2015) guidance for providing instructional 
support for ELLs. 
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Research Methods 


This section describes our research methods by phase, including data sources used. Additional 
details are provided in Appendix A. 

Phase 1 Methods 

District Selection 

All Phase 1 research questions refer to a purposive sample of 10 Massachusetts districts. In this 
section we describe the criteria used to select the 10 districts. Together, these districts represent 
36,944 ELL students, or 45.5 percent of ELLs served statewide. To the extent possible, 
throughout the report we provide comparisons to other student subpopulations in Massachusetts 
where relevant (e.g., never ELLs, former ELLs, and the general student population) to 
contextualize our Phase 1 findings. 

■ Concentration of ELLs. The first five districts have the highest districtwide 
concentrations of ELLs statewide: Worcester (35.1 percent ELL), Lawrence (29.9 
percent), Boston (29.8 percent), Holyoke (28.5 percent), and Lowell (26.6 percent). 6 

■ Innovative ELL Programming. Brockton, the seventh-largest ELL district in the State 
(20 percent of students are ELLs), was selected for inclusion in the Phase 1 analysis 
because of its innovative programming for ELLs (see MBESE, 2009). Brockton serves 
3,441 ELLs, or 4.2 percent of the State’s ELLs. 

More specifically, in the Brockton school district, 7 elementary school ELLs performing 
at levels 1, 2, and 3 on the statewide English proficiency assessment are placed in a 
structured English immersion self-contained setting clustered by language group. These 
classes are taught by dual-licensed teachers who speak the home languages of the 
students in their class. Once ELLs reach levels 4 and 5 on the assessment, they are moved 
into a structured English immersion classroom in their neighborhood school, and English 
as a second language (ESL) instruction is provided by an ESL teacher. 

At the middle and high school levels, ELL instructional models include English as a 
second language, structured English immersion, and transitional bilingual education 
classes. Students performing at levels 1, 2, and 3 on the English proficiency assessment 
have two to three blocks of transitional bilingual education in a self-contained setting 
arranged by language groups. For ELLs who have had interrupted formal education, the 
high school offers special courses such as literacy math, literacy social studies, and 
literacy science. The high school also offers a medical interpretation program for students 
who are proficient in their home language. 

■ Districtwide Concentration of Low-Income Households and ELL Growth in English 
Language Proficiency. The remaining four districts in the Phase 1 analysis were selected 
on the basis of districtwide concentrations of students from low-income households and 
growth in English language proficiency. Appendix A provides more infonnation about 

6 These five districts and corresponding values are based on the 2014-15 Selected Populations Report from ESE 
Profiles, available at http://profiles.doe.mass.edu/state report/selectedpopulations.aspx . 

7 Personal communication with personnel at the Brockton Public Schools ESL/Bilingual services department. 
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the selection of the four prototypical districts shown in Table 1. These four districts serve 
an additional 2,249 ELL students, or 3 percent of the State’s ELL population. 


Table 1. Prototypical Districts Representing Various Concentrations of Students From 
Low-Income Households and ELL Growth in English Language Proficiency 


District 

Characteristics 

Number of Students From 
Low-Income Households 

English Language Proficiency 
Growth 

Weymouth 

Low 

High 

Wachusett 

Low 

Low 

Quincy 

High 

High 

Fall River 

High 

Low 


Data Analysis 

For this phase of the analysis, we drew primarily from the district-level extant data maintained in 
the publicly available District Analysis Review Tools (DART) Detail: ELLs and the School and 
District Profiles, which includes aggregate Massachusetts student demographic, assessment, 
student support, educator, financial, and achievement gap data. We developed charts and tables 
of descriptive statistics to summarize information across selected districts for 2013-14, the most 
recent year of data available. To address research questions (RQs) lc (language learning 
programs) and Id (reclassification guidelines), we conducted short telephone interviews with an 
ELL administrator in each of the 10 selected districts to clarify district policies and procedures. 
Appendix A provides the list of questions we asked each district administrator. Finally, we 
conducted a review of websites and documents related to State policies and practices for ELL 
reclassification. 

Phase 2 Methods 

Data Construction 

To address the Phase 2 research questions, we constructed a statewide longitudinal data set that 
followed a cohort of ELLs over time, beginning with those in kindergarten in Massachusetts 
public schools in the fall of 2003. Our data set consisted of student-level demographics and 
enrollment from the student information management systems (SIMS), standardized English 
language proficiency data from MEPA (fall 2004-spring 2012) and ACCESS for ELLs (spring 
2013-present) assessments, 8 and content-area performance data in mathematics and ELA from 
the MCAS assessments (spring 2006-present). 

Each student in the State has a unique student identifier that remains with him or her over time 
and is common across data sources even if the student changes schools or districts within the 
State. This identifier allowed us to link student records across multiple years, regardless of 
intrastate mobility or retention. In the case of the ELL cohort, tracking outcomes longitudinally 

s Beginning in spring 2013, the ACCESS replaced the MEPA as the statewide English language proficiency 
assessment. We used the ESE-developed MEPA/ACCESS crosswalk to examine outcomes across the transition 
from MEPA to ACCESS during the 2011-12 and 2012-13 transition period. 
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over time will help ESE better understand the educational trajectories of all students who were 
ever classified as ELL, or “ever-ELLs.” Many students in this cohort were reclassified during the 
period of analysis. Table 2 summarizes the school years that 2003-04 cohort ELL students 
entered each grade for those who progressed on time through school and the data we drew on to 
analyze their achievement and progress. 


Table 2. Key Outcomes and School Years Included in the Analysis 



2003 

-04 

2004 

-05 

2005 

-06 

2006 

-07 

2007 

-08 

2008 

-09 

2009 

-10 

2010 

-11 

2011 

-12 

2012 

-13 

2013 

-14 

Grade 

K 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Outcome 

MEPA/ACCESS proficiency 
(RQ2a-l) 












SIMS program exit (RQ2a-2) 












MCAS proficiency (RQ2c) 




• 

• 

• 

• 

• 

• 


• 

On-time promotion (RQ2e) 






• 



• 




Outcomes of Interest 

Proficiency on statewide English proficiency assessments (RQ2a-l; RQ2c). We included 
students’ English language proficiency data over time in our data set as measured by the MEPA 
(fall 2004-spring 2012) and ACCESS for ELLs (spring 2013-present). Consistent with state 
guidelines over the past decade (see ESE, 2005, 2009, 2013), we considered a student to have 
reached proficiency at the following levels across the 10 years of our longitudinal data set: (1) 
Level 4 on MEPA during the 2006-2007 and 2007-2008 test administrations, (2) Level 5 on 
MEPA during the 2008-09 through 2011-12 school years, and (3) Level 5 or above on ACCESS 
during the 2012-13 and 2013-14 test administrations. 

It is important to note that the standard for reaching English language proficiency as measured 
by the statewide English language proficiency assessment changed in several important ways 
during the 10-year study period. At the start of the period of analysis, language proficiency was 
not included on a statewide assessment for ELLs in early grades (Grades K, 1, and 2). 
Consequently, the 2006-07 school year—corresponding to the third grade for students who 
proceeded on time through school—marked the first year in which students in our study cohort 
could demonstrate a score of proficient on a statewide language assessment. 

During its tenure as the statewide language proficiency assessment, the Massachusetts English 
Proficiency Assessment (MEPA) had two iterations. The first iteration (2006-07 and 2007-08 
for our sample) classified students into four proficiency levels, with Level 4 representing 
proficiency. During the second iteration of MEPA (2008-09 and 2011-12), students were 
classified into five proficiency levels, with Level 5 representing proficiency. Beginning in 2012- 
13, Massachusetts began using the ACCESS for ELLs assessment, which classifies students into 
six levels of proficiency. We account for these changes in language proficiency assessments and 
corresponding benchmarks for proficiency in our analysis. 
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Reclassification into mainstream classrooms (RQ2a-2). For ELLs, the SIMS data set includes 
information on whether a student receives ELL instructional services (i.e., LEP = 1; 
reclassification = 0) or whether a student exited ELL services (i.e., LEP = 0; reclassification = 1). 
In our analysis, reclassification is a time-varying variable because ELLs make progress in their 
language development and most will reach a point at which they are considered ready to exit ELL 
instructional services or be reclassified as former ELL (FELL). For the purposes of this analysis, 
we examined only the first occurrence of reclassification and proficiency on the statewide English 
language proficiency assessment. In Massachusetts, reclassification reviews typically occur in the 
spring of each year so that any instructional changes are effective the following fall. We have 
coded our outcome data to reflect these instructional decision-making procedures. 

Proficiency on statewide content-area assessments (RQ2c; RQ2d). We relied on the State-defined 
proficiency levels on the statewide content-area assessment (MCAS) each year during the period 
of analysis in English language arts and mathematics to examine the proportion of students in the 
cohort that scored proficient in the respective assessments while classified as ELL and after exit. 

On-time promotion (RQ2e). We relied on the grade-level enrollment data maintained in the 
SIMS database to calculate a promotion variable for each year of our longitudinal data set. 
Students who progressed on time from one grade to the next were counted as promoted for each 
respective grade transition (promoted =1). Students whose grade level records indicated that 
they were in the same grade for two consecutive years were counted as retained in grade (i.e., 
promoted = 0) for the respective year. We relied on this variable to report the percentage of 
students in the 2003-04 ELL cohort who were promoted on time in fifth and eighth grades. 

District variation in time-to-reclassification and proficiency (RQ2f). The SIMS data set tracks 
students’ districts and schools over time using district and school identification codes, regardless 
of student mobility across the State. We rely on these district and school identification codes to 
examine variation in time-to-reclassification and time-to-proficiency across the 10 sample 
districts. 

Analytic Methods 

To answer RQ2a and RQ2f (please refer to the section Study Questions for a complete list of 
research questions), we relied on discrete time survival analysis to examine the time it takes 
students in the 2003-04 ELL cohort to perform at the proficient level on the statewide English 
language proficiency assessment (RQ2a-l), the time it takes to be reclassified as English 
proficient (RQ2a-2), and how the time-to-reclassification varies by district (RQ2f). We used 
correlational analyses to address the relationship between ELLs’ perfonnance on the statewide 
English language proficiency assessment and the mathematics and English language arts content- 
area assessments, and we used descriptive statistics to examine outcomes over time (RQ2b, 
RQ2d). Phase 2 data and analysis methods are described in greater detail in Appendix A. In the 
following sections, we provide an overview of key outcome and predictor variables as they relate 
to each data source and research question. 
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Phase 1 Findings 

Phase 1 provides an aggregate district-level analysis of the ELL student demographic, 
geographic, and academic program profiles that aims to document key trends across a 
purposively selected sample of 10 Massachusetts districts. As noted in the Research Methods 
section, the districts were selected on the basis of three criteria: (1) districts with the highest 
concentrations of ELLs statewide (Worcester, Lawrence, Boston, Holyoke, and Lowell), (2) 
districts with innovative ELL programming (Brockton), and (3) prototypical districts 
representing four combinations of students from low- and high-income households and students 
with low and high growth in English language proficiency (see Table 1; Weymouth, Wachusett, 
Quincy, and Pall River). We organize the reporting of Phase 1 findings by these groupings. 

We begin with a portrait of these 10 districts’ key ELL demographics (RQla) and the distribution of 
ELLs across schools in the districts (RQlb). Next, we examine the language services for ELLs in 
these districts, as well as ELLs’ academic and nonacademic profiles (RQlc). Last, we provide a 
description of these districts’ guidelines for making reclassification decisions (RQld). 

RQla. ELL Key Demographics 


Key finding 1—ELL growth: All sampled districts experienced substantial growth in their ELL populations 
over the last decade, with the highest growth districts also among those with the highest concentrations of ELLs 
in the State—Worcester (from a student body that was 13.5 percent ELL in 2003-04 to 31.7 percent in 2013-14), 
Lawrence (from 16.6 percent ELL to 28.2 percent ELL), Boston (from 19 percent ELL to 29.9 percent ELL), and 
Brockton (from 7.4 percent ELL to 20 percent ELL). 

Key finding 2—Linguistic diversity: The sampled districts have large clustering of speakers of a common home 
language: six of the 10 districts have more than 50 percent of ELLs who speak a common language (Worcester, 
Lawrence, Boston, Holyoke, Brockton, Fall River), whereas the second most common language in those same 
districts is spoken by fewer than 2 percent of ELLs in the district. 

Key finding 3—Country of origin: In all sampled districts, the most common country of origin was the United 
States—the majority of ELLs in all sampled districts are U.S.-born (this figure includes Puerto Rican students). 
Statewide, 80 percent of ELLs are U.S.-born and in some districts, the proportion is as high as 99 percent (e.g., 
Holyoke). 

Key finding 4—New-to-Massachusetts students: More than one in four ELLs are new to Massachusetts 9 across 
all sampled districts. The two districts with the highest proportion of new-to-Massachusetts students are both 
prototypical districts selected for their ELLs’ high growth in English language proficiency—Weymouth (60 
percent) and Quincy (57.2 percent). 

Key finding 5—Speciai education representation: Across the 10 sampled districts, we generally observe 
similar rates of special education identification among ELLs and the general population. However, in Holyoke, 
ELLs appear to be identified for special education services at rates higher than those in the general population 
(33.6 percent versus 25.1 percent) when compared to the discrepancy between ELL and general population 
special education identification in the other sampled districts. ELLs and non-ELLs in Holyoke are identified at 
higher rates than in the state. In Quincy, ELLs appear to be identified at rates lower than those in the overall 
population (4.3 percent versus 15.5 percent) as well as statewide. Lawrence, Brockton, and Weymouth also had a 
somewhat lower percentage of ELLs receiving special education services than in the overall student population. 


9 It is not clear from the way this variable is measured whether these students are also new to the United States or 
have transferred from another state. We refer to these students as “new-to-Massachusetts” to differentiate from the 
term “newcomers”—typically referring to students who are new arrivals to the United States. 
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This section discusses key demographics of the ELLs in the Phase 1 district sample. Figure 1 shows 
the overall percentage of students who are ELLs in our sample of districts in 2003-04 (baseline) and 
in 2013-14 (the latest year of available data). The high numbers of ELLs currently enrolled in many 
of these districts are unsurprising given that our sample was selected purposively to include several 
districts with this characteristic. However, some of the districts have experienced high levels of 
growth in ELL enrollment since 2003-04, especially Worcester, Lawrence, Boston, and Brockton— 
four districts that are also among those that have the highest districtwide concentrations of ELLs. 


Figure 1. Percentage of ELLs in Sampled Districts (2003-04 and 2013-14) 
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Note. ELP = English language proficiency. 


Next, we examined home language and country of origin. Table 3 displays the first and second 
most common non-English languages spoken by ELLs. Spanish is the most common non- 
English language in most districts except three of the 10 sampled districts—Brockton, Quincy, 
and Weymouth, whose largest language group among ELLs includes Cape Verdean (n = 1,893; 
55.8 percent), Mandarin (n = 380; 28.1 percent), and Portuguese (n = 90; 41.9 percent), 
respectively. Although all the districts in the sample have speakers of multiple languages 
(ranging from 14 languages in Holyoke to 80 languages in Worcester), in all cases the number 
and percentage of speakers of the most common non-English language are substantially higher 
than the next most common non-English language. For instance, six of the 10 districts have more 
than 50 percent of ELLs who speak a common language (Worcester, Lawrence, Boston, 

Holyoke, Brockton, Fall River), whereas the second most common language in those same 
districts is spoken by fewer than 2 percent of ELLs in the district. This clustering of speakers of 
the same home language as well as the overall linguistic diversity across districts has 
implications for dual language programming, as we note in the Discussion section of the report. 
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Table 3. First and Second Most Common Languages Spoken by Massachusetts ELLs in 
Sampled Districts and Statewide (2013-14) 


District 

Total 

Number of 
Languages 

Most Common Language 
(Number of Speakers; 
Percentage of District’s ELLs) 

Second Most Common Language 
(Number of Speakers; Percentage 
of District’s ELLs) 

Worcester 

80 

Spanish (4,716; 61%) 

Vietnamese (565; 1%) 

Lawrence 

16 

Spanish (3,751; 98%) 

Khmer/Khmai (16; 0.4%) 

Boston 

74 

Spanish (9,592; 59%) 

Haitian Creole (1,571; 1%) 

Holyoke 

14 

Spanish (1,601; 99%) 

Kirundi (5; 0.3%) 

Lowell 

40 

Spanish (1,529; 37%) 

Khmer/Khmai (1,477; 4%) 

Brockton 

37 

Cape Verdean (1,893; 56%) 

Haitian Creole (680; 2%) 

Weymouth 

25 

Portuguese (90; 42%) 

Arabic (33; 2%) 

Wachusett 

28 

Spanish (19; 19%) 

Chinese-Other (11; 1%) 

Quincy 

50 

Mandarin (380; 28%) 

Canton (300; 2%) 

Fall River 

19 

Spanish (570; 65%) 

Portuguese (199; 2%) 

Massachusetts 

123 

Spanish (41,058; 54%) 

Portuguese (5,277; 7%) 


Note. Percentages rounded to nearest whole number. 


Table 4 displays the first and second most common countries of origin for Massachusetts ELLs. 
The most common country of origin for ELLs is the United States for all districts in the sample, 
and the number and percentage of students from the United States are substantially higher than 
the next most common country of origin in all districts. It is likely that a portion of these students 
are from Puerto Rico, but we cannot determine how many; the data do not differentiate students 
from Massachusetts and other U.S. states or territories. 


Table 4. First and Second Most Common Countries of Origin of Massachusetts ELLs in 
Sampled Districts and Statewide (2013-14) 


District 

Total Number of 
Countries of Origin 

Most Common Country of 
Origin (Number of Students; 
Percentage of District’s ELLs) 

Second Most Common Country of 
Origin (Number of Students; 
Percentage of District’s ELLs) 

Worcester 

25 

USA (7,404; 95%) 

Ghana (140; 2%) 

Lawrence 

20 

USA (2,277; 60%) 

Dominican Republic (1,442; 

38%) 

Boston 

77 

USA (13,274; 82%) 

Dominican Republic (698; 4%) 

Holyoke 

3 

USA (1,623; 99.9%) 

Dominican Republic (1; 0.1%) 

Lowell 

55 

USA (3,583; 87%) 

Cambodia (83; 2%) 

Brockton 

31 

USA (2,227; 66%) 

Cape Verde (749; 22%) 

Weymouth 

18 

USA (159; 74%) 

Brazil (13; 6%) 

Wachusett 

5 

USA (95; 94%) 

India (2; 2%) 

Quincy 

47 

USA (900; 67%) 

China (279; 21%) 

Fall River 

23 

USA (751; 86%) 

Portugal (41; 5%) 

Massachusetts 

159 

USA (60,345; 80%) 

Dominican Republic (3,142; 4%) 
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In addition to collecting data on the country of origin for Massachusetts ELLs, the state also 
identifies students who are in their first or second year in a Massachusetts public school. In 
Figure 2, we display the percentage of ELLs 10 in Grades 2-12 who are identified as recent 
arrivals. It is important to note that this statistic is not an equivalent measure to time in the 
United States because these figures include students who have emigrated from another country 
as well as students who have transferred to a Massachusetts public school from another state or a 
private school. Three of the districts in the sample (Lawrence, Quincy, and Weymouth) have 
ELL populations in which more than half of the students are in the first or second year in a 
Massachusetts public school. The other districts have percentages of recent ELL arrivals ranging 
from 27.3 percent (Lowell) to 40.7 percent (Boston). 


Figure 2. Percentage of ELLs in Grades 2-12 Who Are in the First or Second Year in 
Massachusetts Public Schools in Sampled Districts (2013-14) 



We also examined whether ELLs belong to other state-defined selected student populations, 
including those from low-income households and those receiving special education services. 
Figure 3 displays the percentage of ELLs in each of the 10 sampled districts who were from low- 
income households in 2013-14 (light gray bars). These figures are compared to the overall 
percentages of students from low-income households in each respective district in 2013-14 (dark 
gray bars). Note that ELLs from low-income households comprised part of the overall totals in 
these districts. Nonetheless, the percentage of ELLs who were from low-income households was 
higher than the overall percentage of students from low-income households in the State (overlaid 
line). This difference is reflected in the comparison of State percentages of ELLs from low- 
income households (80.5 percent) and overall percentage of students from low-income 
households (38.3 percent) in 2013-14, although the disparities in the selected districts are lower, 


10 The DART includes only ELLs in Grades 2-12 in order to avoid artificially inflating the indicator, because 
younger students are in their first or second year of school by definition (ESE, 2014a). 


American Institutes for Research 


Massachusetts English Language Learners’ Profiles and Progress—12 




likely because of the high numbers of ELLs contributing to the overall percentages of students 
from low-income households. 


Figure 3. Percentage of Students From Low-Income Households Among ELLs in Sampled 
Districts, All Students in Sampled Districts, and ELLs Statewide (2013-14) 



Note. ELLs are also included in the statewide totals. 


In most of the districts in the sample, the proportion of ELLs who were from low-income 
households exceeded the State average of ELLs from low-income families (80.5 percent). 
However, in several of the sampled districts, the proportion of ELLs who were from low-income 
households was lower than the State average: Quincy (74.6 percent), Weymouth (57.2 percent), 
and Wachusett (26.7 percent). 

Figure 4 shows the respective proportions of ELLs and students in the overall population who 
received special education services, both statewide (overlaid line) and in the 10 selected districts 
in 2013-14 (sets of gray bars). Examining the proportion of ELLs statewide receiving special 
education services in relation to the overall student population reveals comparable rates of 
services (16.5 percent compared to 16.4 percent, respectively). Across the 10 sampled districts, 
we generally observe similar rates of special education identification among ELLs and the 
general population. However, in Holyoke, ELLs appear to be identified for special education 
services at higher rates than in the general population (33.6 percent compared to 25.1 percent) 
when compared to this same difference in the other sampled districts. ELLs and non-ELLs in 
Holyoke are identified at higher rates than in the State. In Quincy, ELLs appear to be identified 
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at lower rates than in the overall population (4.3 percent compared to 15.5 percent) as well as the 
statewide rates of identification. Lawrence, Brockton, and Weymouth also had a somewhat lower 
percentage of ELLs receiving special education services than in the overall student population. 

Figure 4. Percentage of ELLs Receiving Special Education Services in Sampled Districts 
Compared With Percentage of All Students Receiving Special Education Services in 
Massachusetts (2013-14) 
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RQlb. Distribution of ELLs in Schools 


Key finding 6—Segregation: The average ELL in Massachusetts attends "triply segregated” schools—those 
with high proportions of students from low-income households and minority and English language learner 
classmates—compared to the student composition of the average non-ELL’s school (see Figure 6). 


The previous section discussed district-level ELL demographics in a purposively selected sample 
of districts. To contextualize these findings further, in this section we discuss characteristics of 
the schools Massachusetts ELLs attend, starting with the distribution of ELLs across different 
school levels (e.g., elementary, middle, secondary). An extensive statewide profile of ELLs 
(ESE, 2012) found that ELLs tend to be clustered in the elementary grades—in the 2010-11 
school year, 37.6 percent of Massachusetts ELLs were enrolled in Grades PK-2 and an 
additional 26.4 percent were enrolled in Grades 3-5. Figure 5 shows the percentage of ELL 
students in the State attending schools at each school level in 2013-14. Consistent with the 
earlier study, ELLs formed a higher proportion of elementary school students (11.8 percent) than 
middle school students (6.5 percent) or secondary school students (4.9 percent), with the 
proportion of ELLs in elementary schools more than twice as high as the proportion of ELLs in 
secondary schools. 
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Figure 5. Percentage of ELL Students at Each School Level (Elementary, Middle, 
Secondary) in Massachusetts (2013-14) 
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A comparison of the demographic characteristics of schools attended by non-ELLs and ELLs 
reveals that on average, ELLs attend schools with higher concentrations of students from low- 
income households, minority, and other LEP students than do non-ELLs. Figure 6 shows that the 
average non-ELL student attends a school that has an average of 40.1 percent of students from 
low-income households while the average ELL student attends a school where 74.3 percent of 
the students are from low-income households. The average non-ELL attends a school where, on 
average, 15 percent and 8 percent of the students are Hispanic and Black, respectively, compared 
to 39.7 percent and 16.9 percent of the respective student populations at the average school 
attended by ELLs. In addition, the average non-ELL attends a school where 6.3 percent of 
classmates are LEP compared to 26.3 percent of classmates for the ELL group. This “triple 
segregation” of ELL students by income, minority, and LEP status (Orfield, 2001; Orfield & 

Lee, 2006; Rios-Aguilar & Gandara, 2012) has been well documented across U.S. schools. 

Note that on average, non-ELLs and ELLs attend schools with equivalent proportions of students 
receiving special education services—the average non-ELL attends a school where 17.5 percent 
of classmates receive services compared to 17.2 percent for ELLs. 
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Figure 6. Demographic Characteristics of Schools Attended by Non-ELL and ELL 
Students in Sampled Districts, by Characteristic (2013-14) 
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RQlc. ELL Academic and Nonacademic Profiles 

Thus far in this report, we have focused on demographic characteristics of ELLs in 10 select 
districts in Massachusetts and the schools they attend. RQlc in Phase 1 focuses on instructional 
services ELLs receive and descriptions of their academic and nonacademic outcomes, focusing 
again on the 10 purposively sampled districts. 

ELL Instructional Services _ 

Key finding 7—Teacher ELL qualifications: Available data 11 indicate that districts lack personnel with 
specialized credentials to implement ELL instructional services—nearly every district in our sample has a very 
low ratio of teachers classified as providing support to ELLs relative to the proportion of ELLs. For example, in 
Worcester, 31.7 percent of students are ELL, but there are no teachers reported with ESL, bilingual or dual 
language certifications. 


As noted in the Introduction, Massachusetts law requires that all ELLs be educated in an SEI 
program, unless students are enrolled in a dual language instruction program (two-way bilingual) 
or the student receives a waiver. SEI is therefore the default program design for ELLs throughout 


11 The DART tool includes the total number of ESL full-time equivalents (FTEs) reported at the district and school levels and the 
number of teachers providing sheltered content instruction or other bilingual education instruction for ELL students. Sheltered 
content and other bilingual teacher FTEs include all instructional staff reported as having a job classification of teacher, co¬ 
teacher, or support content instmction teacher and whose work assignment includes Sheltered Content Teacher or Bilingual 
Education. The DART notes that ELL programming decisions vary widely district to district, and this variation may be reflected 
in the allocation of resources between schools. For example, some ESL teachers may be assigned at the district level and work 
with students across multiple schools, while in other districts, ESL teachers are assigned at the school level. Note that RETELL 
data are not yet included in the DART and thus are not reported here. Please refer to the DART User Guide for more information: 
http://www.mass.gov/edu/docs/ese/accountabilitv/dart/dart-user-guide.pdf 
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the State, as reflected in Table 5, which shows reported ELL programming for 2013-14 in the 
study’s Phase 1 10 selected districts. Some districts, including Boston, Brockton, and Holyoke, 
also offer dual language instruction, but enrollment in these programs is low (less than 3 
percent). According to the DART data, a relatively high percentage of ELLs in Weymouth opted 
out of ELL services (7.4 percent), and nearly one fifth of ELLs (18.2 percent) are not enrolled in 
any language learning program in Worcester. 12 


Table 5. Enrollment of ELLs in Language Learning Programs, by District (2013-14) 


District 

n 

Sheltered 

English 

Immersion 

Dual 

Language 

Instruction 

Opted Out 

Other 

No 

Program 

Worcester 

7,780 

79.1% 

0.0% 

1.2% 

1.5% 

18.2%* 

Lawrence 

3,813 

100.0% 

0.0% 

0.0% 

0.0% 

0.0% 

Boston 

16,239 

95.6% 

3.0% 

0.0% 

1.4% 

0.0% 

Holyoke 

1,625 

98.7% 

0.1% 

1.2% 

0.0% 

0.1% 

Lowell 

4,121 

98.3% 

0.0% 

1.6% 

0.1% 

0.0% 

Brockton 

3,395 

79.4% 

2.1% 

1.7% 

15.4% 

1.4% 

Weymouth 

215 

92.6% 

0.0% 

7.4% 

0.0% 

0.0% 

Wachusett 

101 

99.0% 

0.0% 

0.0% 

0.0% 

1.0% 

Quincy 

1,350 

99.9% 

0.0% 

0.1% 

0.0% 

0.0% 

Fall River 

872 

98.9% 

0.0% 

1.0% 

0.1% 

0.0% 

Massachusetts 

75,947 

92.8% 

1.2% 

1.8% 

1.6% 

2.6% 


Note. The high proportion of students enrolled in “no program” in Worcester may be a result of ELL enrollment in 
kindergarten services where no specific program type is offered. 


We also examined the numbers of teachers classified as providing SEI, ESL, and other language 
support to ELLs by district. These data are presented in Table 6. Staffing data are reported by 
districts, and districts vary in how they code teacher work assignments, so the data are not 
directly comparable across districts (ESE, 2014a). However, when we compare the percentage of 
ELL-related staffing classifications to the percentage of ELL students within each district, we 
find that nearly every district in our sample has a very low ratio of teachers classified as 
providing support to ELLs (Weymouth has the highest ratio, with 1.4 percent of its staff 
classified as ESL teachers and 3.1 percent ELL students). Wachusett and Worcester report no 
teachers providing ELL support; these districts report ELL support provided only by bilingual 
paraprofessionals. This staffing configuration is particularly notable for Worcester, which has 
31.7 percent ELL students. 


12 The Worcester nonenrollment data may be a result of a lack of specific programs for kindergarten students. 
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Table 6. Teachers Classified as Providing Language Support to ELLs, by Sampled Districts 
(2013-14) 


District 

Percent 

ELLs 

ELL 

Directors 

Total 

District 

Teachers 

(FTE) 

Percent 

ESL 

Teachers 

Percent 

Sheltered 

Content 

Teachers 

Percent 

Bilingual 

Teachers 

Bilingual 

Paraprofessionals 

Worcester 

31.7% 

1 

1,413 

0.0% 

0.0% 

0.0% 

37 

Lawrence 

28.2% 

1 

1,052 

4.7% 

0.7% 

0.0% 

13 

Boston 

29.9% 

1 

4,001 

4.4% 

0.0% 

0.0% 

0 

Holyoke 

29.2% 

0 

500 

8.3% 

0.2% 

0.0% 

0 

Lowell 

29.4% 

0 

984 

2.5% 

1.6% 

0.0% 

26 

Brockton 

20.0% 

4 

1,113 

2.4% 

0.0% 

1.1% 

31 

Weymouth 

3.1% 

0 

433 

1.4% 

0.0% 

0.0% 

1 

Wachusett 

1.3% 

1 

456 

0.0% 

0.0% 

0.0% 

1 

Quincy 

14.5% 

1 

679 

5.4% 

0.0% 

0.0% 

4 

Fall River 

8.5% 

1 

717 

0.1% 

1.7% 

0.0% 

0 

Massachusetts 

7.7% 

68 

70,489 

1.9% 

0.2% 

0.2% 

289 


Note. This table only includes currently available statewide data as reported in the DART. Data on the RETELL 
initiative are not included here. 


However, due to the way in which programming and teacher work assignments are reported to 
the State, the data in this section are likely to obscure additional supports provided to ELLs. Lor 
example, ESL-licensed teachers who provide services to ELLs for a portion of the school day 
would not be classified as the teachers of record and therefore may not appear in Table 6. 
Lurther, no systematic, statewide data on a major initiative launched in fall 2012 to train teachers 
to instruct ELLs—Rethinking Equity and Teaching for English Language Learners, or 
RETELL 13 —is yet available. Under this initiative, all core academic teachers in the State will be 
required to complete comprehensive professional development in sheltered English instruction 
(SEI) methods by July 1, 2016, and teachers entering the school system after that date must have 
the endorsement. Thus, the number of teachers with training to teach ELLs is inevitably higher. 

In the following sections, we report ELL student outcomes in the 10 sampled districts on the 
2013-14 English language proficiency assessment, ACCESS, as well as the most recent 
administration of the statewide English language arts and mathematics assessments. Whenever 
possible, we compare performance for students in our 10-district sample on these assessments to 
statewide performance for ELLs and, when relevant, to all students 14 (i.e., on content-area 
assessments only). 


13 For additional information on the RETELL rollout, see http://www.doe.mass.edu/retell/2012-Q2-27bese.pdf. 

14 Note that ELLs in the 10 sampled districts are also included in statewide ELL and overall student averages. 
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ELL Academic Outcomes in Selected Districts 


Key finding 8—Language proficiency: Overall performance on ACCESS is low in many districts, but 90 
percent of the sampled districts are making progress on the State English language proficiency assessment: four 
of the sampled districts have less than 10 percent of ELLs who scored proficient on ACCESS. However, more 
than half of ELLs in nine out of the 10 sampled districts are making progress (by the State definition) on the ELP 
assessment, particularly at the lowest levels of language proficiency. 

Key finding 9—Content-area proficiency: The percentage of ELLs performing at proficient or higher on the 
ELA, mathematics, and science MCAS in all 10 districts falls below the State average for all students. However, 
there are two notable exceptions to this trend: the percentage of ELLs performing at proficient or higher on the 
mathematics MCAS in Quincy and Wachusett—56 percent and 58 percent, respectively—approaches the 
statewide average of 60 percent. ELLs in eight out of 10 of the sampled districts showed growth that surpassed 
about 50 percent of ELLs statewide with similar MCAS histories. 

Key finding 10—Retention rates: All districts except Lowell and Quincy had higher ELL retention in grade 
rates than the State overall average of 1.6 percent in 2013-14; Brockton, Lawrence, Holyoke, and Boston had 
higher retention rates than the State ELL average of 3.6 percent. 

Key finding 11—Dropout rates: All districts had higher ELL dropout rates than the State overall average of 2.0 
percent in 2013-14. Boston, Brockton, Holyoke, and Wachusett all exceeded the State ELL dropout rate of 6.5 
percent. 

Key finding 12—Graduation rates: All districts had lower ELL four-year graduation rates than the State overall 
rate of 86.1 percent. Weymouth, Worcester, and Quincy had higher ELL four-year graduation rates than the State 
ELL average of 63.9 percent. 


ELL Performance on English Language Proficiency Assessments 

We examined ELLs’ outcomes on the ACCESS for ELLs English language proficiency 
assessment in the 10 selected districts. Only students who are currently classified as ELL 15 
would be included in these results; once students exit ELL services, they are no longer assessed 
on the ACCESS for ELLs. Results by number of years ELLs have been in Massachusetts schools 
are provided in Table B1 in Appendix B. 

Figure 7 compares the percentage of ELLs scoring proficient or higher and making progress 16 on 
the ACCESS for ELLs assessment (2013-14) in each of the 10 selected districts. With respect to 
the percentage of ELLs scoring proficient on ACCESS, in most districts, approximately 20 
percent to 25 percent of ELLs scored proficient or higher. Several districts stand out as falling 
above or below this mark. In Weymouth—included in the sample for its high proficiency 
growth—31 percent of ELLs scored at least proficient on ACCESS. Four of the sampled districts 
had much smaller proportions of ELLs that scored proficient on ACCESS—Brockton (16 
percent proficient), Fall River (16 percent proficient), Lawrence (10 percent proficient), and 


15 Students who must participate in the ACCESS for ELLs test program include ELL students who were reported as LLP in 
October SIMS and ELL students who enroll in a Massachusetts school after the fall SIMS submission who will be reported as 
LEP in the March 2016 SIMS. Students who are reported as ELL in October but are exited from ELL status before the beginning 
of the testing window (i.e., before January of the given year) are not required to participate. Testing participation requirements 
are available at http://www.doe.mass.edu/mcas/participation/?section=ell. 

16 The progress measure was calculated by using a student’s actual 2012 MEPA score and projected 2013 MEPA score based on 
an equivalent percentile linked to the student’s ACCESS scores. After translating these two scores on a common (MEPA) scale, 
the State determined whether a student made progress over time. More infonnation on the State methodology for calculating 
progress is available at http://www.mass.gov/edu/government/departments-and-boards/ese/programs/accountability/tools-and- 
resources/district-analysis-review-and-assistance/. 
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Holyoke (11 percent proficient). With respect to progress, more than half of ELLs are making 
progress in all of the sampled districts with the exception of Holyoke (where 45 percent of ELLs 
are making progress). Weymouth also had the highest percentage of ELLs showing progress 
(87 percent). 


Figure 7. Percentage of ELLs Scoring Proficient or Higher and Making Progress on the 
ACCESS for ELLs Assessment (2013-14), by Sampled District 



■ District ELLs scoring proficient or higher ■ District ELLs making progress 


We also plotted the number of ELLs scoring proficient or higher on the ACCESS in 2013-14 in 
the 10 selected districts by the number of years ELLs had been enrolled in Massachusetts 
schools: 1-2 years, 3-4 years, and 5 or more years (Figure 8). 

Most districts show the highest average proficiency growth in the period between Years 1-2 in 
Massachusetts schools and Years 3-4, followed by moderate growth after students had spent five 
or more years in Massachusetts schools. Exceptions include Wachusett—initially selected as a 
prototypical low-proficiency-growth district—whose ELLs demonstrate higher rates of growth 
on ACCESS after three to four years in Massachusetts schools, and Weymouth—initially 
selected as a prototypical high-growth district—whose ELLs demonstrate higher rates of growth 
on ACCESS in their third and fourth years in Massachusetts schools compared to ELLs who 
spent five or more years in Massachusetts schools. 

Several limitations related to the data presented in Figure 8 should be noted. First, data are 
disaggregated by time in Massachusetts schools and not by proficiency level on ACCESS. 
Second, enrollment in Massachusetts schools in some cases may be a proxy for time in the 
United States but would not be equivalent to time in the United States for students who move 
into Massachusetts schools from out of state. Third, these data are cross-sectional—each time 
“bin” (i.e., 1-2 years, 3-4 years, 5 or more years) represents different groups of students. 
Students who have remained in Massachusetts schools for 5 or more years without being exited 
from ELL programs may represent a select group of students who demonstrate lower 


American Institutes for Research 


Massachusetts English Language Learners’ Profiles and Progress—20 





performance on the statewide English language proficiency assessment, which prevents them 
from exiting ELL instructional services. 

Figure 8. Percentage of ELLs Scoring Proficient or Higher on the ACCESS for ELLs, by 
Number of Years in Massachusetts Schools (2013-14) 


A. High ELL Districts/Innovative Program 
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ELL Performance on Statewide English Language Arts, Mathematics, and Science Assessments 

The majority of ELLs in Massachusetts participate in statewide ELA and mathematics content-area 
assessments—the MCAS—regardless of their enrollment in an ELL instructional program or their 
time in Massachusetts schools. The one exception is for ELLs who are in their first year of ELL 
classification and enrolled after March of the particular test administration; the ELA assessment is 
optional for this group, although the mathematics assessment is still required (ESE, 2015). 

Figure 9 compares the percentage of ELLs scoring proficient or higher on MCAS English 
language arts, mathematics, and science in the 10 selected districts in 2013-14 with the statewide 
average of all students scoring at the proficient level in these areas and the statewide average of 
ELLs scoring proficient in these areas (as overlaid lines). 18 The bars indicating ELLs rates of 
scoring proficient on the MCAS in the sampled district fall below the line denoting the State 
average for all students in ELA (Panel A), mathematics (Panel B), and science (Panel C). 
However, it should be noted that ELLs in Quincy and Wachusett surpass the State ELL average 
percentage proficient in all three subjects and approach the State overall average in mathematics. 
In addition, several of the sampled districts exceeded the State ELL average in one or more 
content-area assessments: Lowell (ELA and mathematics), Worcester (ELA), and Boston 
(mathematics). We include a full listing of proficiency rates by district and ACCESS for ELLs 
assessment level in Table B2 in Appendix B. 


17 Proficient is one of four general achievement levels on the MCAS. The four levels, from lowest to highest, are 
Warning, Needs Improvement, Proficient, and Advanced. 

18 Note that the selected district ELLs are included in both of these averages. This feature is especially important to 
keep in mind for the statewide average of ELLs given that our sample includes six of the State’s seven biggest ELL 
districts. 
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Figure 9. Percentage of ELL Students Scoring Proficient or Higher on the MC AS in English 
Language Arts, Mathematics, and Science, by District, With Percentage of All Massachusetts 
ELLs and All Massachusetts Students Scoring Proficient or Higher Overlaid (2013-14) 
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In addition to measuring student perfonnance relative to grade-level standards, Massachusetts 
measures students’ progress using student growth percentiles (SGPs). SGPs compare changes in 
students’ MCAS scores with changes seen in groups of students with similar perfonnance histories 
and take demographic characteristics into account (ESE, 2011). Figure 10 shows median SGPs for 
ELLs in the 10 selected districts in 2013-14 for English language arts and mathematics (Table B3 in 
Appendix B provides these data by number of years students were enrolled in Massachusetts 
schools). Most of the sampled districts are perfonning at or above the State Median ELL SGPs with 
the exception of Holyoke and Weymouth. For instance, ELL students in Worcester, Lawrence, 
Brockton, and Quincy were at or above an SGP of 50 for English language arts and mathematics, 
indicating that students in these districts grew more than did 50 percent of their “academic peers,” or 
students with similar MCAS score histories. In Boston, ELLs’ SGPs were at the 50th percentile in 
both ELA and mathematics, indicating that students in these districts are perfonning better than 50 
percent of their academic peers in both content areas. 


Figure 10. ELL Median Student Growth Percentile (SGP) in English Language Arts and 
Mathematics, by Sampled District, With Statewide ELL SGPs Overlaid (2013-14) 



SGPs compare students’ academic growth to the growth of other students as defined by prior MCAS 
scores, not by student demographic data. We also sought to compare ELLs’ perfonnance with the 
perfonnance of reclassified ELLs (i.e., fonner ELLs [FELLs]) and students who had never been 
classified as ELLs. Figure 11 makes this comparison using the MCAS Composite Perfonnance Index 
(CPI) for English language arts, mathematics, and science (2013-14); the data are also provided in 
Table B4 in Appendix B). The CPI is a measure of each student’s proximity to a proficient score on 
the respective assessments. Students scoring at the proficient level or above are assigned 100 points 
while students with low MCAS scores are assigned 0 points. Group CPIs provide a snapshot of how 
close particular subgroups are to the proficient mark (the total number of CPI points is summed 
across the group and divided by the number of students in that group; ESE, 2014b). 

ELLs’ average CPI lags behind that of FELLs and students who were never ELLs in every 
subject area and district, with smaller gaps in mathematics than in English language arts. 
However, FELLs outperformed students who were never ELLs in their districts in English 
language arts and mathematics in Worcester, Boston, Holyoke, Lowell, Brockton, and 
Wachusett; in mathematics in Quincy; and in science in Worcester and Brockton. 
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Figure 11. MCAS Composite Performance Index for ELLs, Former ELLs (FELLs), and 
Students Who Were Never ELLs in English Language Arts, Mathematics, and Science, by 
Selected Districts and Statewide (2013-14) 



















In addition to standardized assessment scores, we examined a number of other indicators of 
student academic progress, including grade retention rates (Figure 12), dropout rates (Figure 13), 
and four- and five-year graduation rates (Figure 14). In all cases, we compared ELLs’ outcomes 
in the 10 selected districts to the State overall average rates and statewide ELL rates in the latest 
years for which data were available. 19 

In most of our sampled districts, ELLs exhibit higher grade retention and dropout rates and lower 
four-year graduation rates than the average student in Massachusetts. Across the retention, 
dropout, and graduation indicators, Boston and Holyoke stood out as having the lowest 
perfonnance, underperfonning the State ELL average on all three measures (i.e., higher grade 
retention and dropout rates, and lower four-year graduation rates). However, ELL students in 
Lowell and Quincy showed better outcomes across the three indicators in comparison to state 
average ELL rates. Quincy exhibited higher four- and five-year graduation rates for ELLs than 
the State average four- and five-year graduation rates for all students. 


Figure 12. Retention Rate of ELLs in Massachusetts by Selected Districts, With Statewide 
Retention Rates for ELLs and All Students Overlaid (2013-14) 



19 As with assessment data, the selected districts’ ELLs’ rates were included in the overall State average rates and 
statewide ELL rates. 
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Figure 13. Dropout Rate of ELLs in Massachusetts, by Selected Districts, With Statewide 
Overall Dropout Rate and Statewide ELL Dropout Rate Overlaid (2013-14) 
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Figure 14. Four-Year and Five-Year ELL Graduation Rates, by Selected Districts, With 
State ELL Four- and Five-Year Graduation Rates and Overall State Four- and Five-Year 
Graduation Rates Overlaid (2013-14) 



Note. Graduation rates will not be publicly reported for cohort counts fewer than 6. 
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ELL Nonacademic Outcomes in Selected Districts 


Key finding 13—Chronic absenteeism: All districts except Wachusett and Quincy had higher ELL chronic 
absenteeism rates than the State overall average of 12.3 percent in 2013-14. Brockton’s rate of chronic 
absenteeism is right below the state average (12.1 percent). Boston, Holyoke and Fall River exceeded the State 
ELL average of 17.2 percent. 

Key finding 14—Stability: All districts had lower ELL stability rates than the overall State average of 96.2 
percent in 2013-14. 

Key finding 15—Disciplinary action: All districts except Lawrence, Boston, and Quincy had higher in-school 
suspension rates than the State overall rate of 2.2 percent in 2013-14, and more than half of the districts 
(Worcester, Boston, Holyoke, Lowell, Brockton, and Fall River) exceeded the State overall rate of 4.3 percent 
out-of-school suspensions. 


We examined three nonacademic indicators of achievement in the 10 selected districts, including 
chronic absenteeism, stability, and disciplinary actions. 

Massachusetts defines chronic absenteeism as a rate of absenteeism more than 10 percent of a 
student’s total days enrolled, indicating extended periods of absence. Figure 15 compares chronic 
absenteeism rates among ELLs in the 10 selected districts to the statewide ELL chronic absenteeism 
rate and overall State rate of chronic absenteeism in 2013-14 (the latest year for which data were 
available).' Three districts—Brockton, Quincy and Wachusett—have rates of ELL chronic 
absenteeism below the overall State chronic absenteeism rate, and Worcester, Lawrence, Lowell, 
and Weymouth are below the State ELL rate. On the other hand, Boston, Holyoke, and Fall River 
show rates of chronic absenteeism far above the State average for all students and for ELLs. 

Figure 15. ELLs’ Chronic Absenteeism Rate, by Selected Districts, With State ELL Chronic 
Absenteeism Rate and Overall State Chronic Absenteeism Rate Overlaid (2013-14) 


40 

35 



20 The ELLs included in the district rates are also included in the overall State rates. 
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Figure 16 shows ELLs’ stability rates, with comparisons to State ELL and overall stability rates 
in 2013-14 21 The stability rate is the proportion of students who remain enrolled within a district 
during the course of a school year and is an indicator of student mobility (although it does not 
capture between-year mobility). None of the districts in our sample had an ELL stability rate 
above 96.2 percent, the State overall average. Only one district—Wachusett—had better stability 
rates than the ELL State average of 90.9 percent. 

Figure 16. ELLs’ Stability Rate, by Selected Districts, With State ELL Stability Rate and 
Overall State Stability Rate Overlaid (2013-14) 


Massachusetts, 96.2 



Finally, we examined in- and out-of-school suspension rates for ELLs, comparing those rates to 
State averages for all students and for ELLs (Figure 17). These rates indicate the number of 
students who received in- or out-of-school suspensions but do not indicate how many incidents 
occurred per student nor the severity of the incidents. In general, Boston and Quincy exhibited 
lower rates than the State averages. Brockton and Holyoke stand out as having rates of out-of¬ 
school suspensions (11.8 percent and 20.3 percent, respectively) that are higher than the State 
ELL average out-of-school suspension rate. It is noteworthy that the State ELL out-of-school 
suspension rate (8.3 percent) is itself close to two times higher than the State overall average of 
4.3 percent. In-school suspension rates tended to be lower than out-of-school suspension rates in 
all instances except in Lawrence, which had a slightly higher in-school suspension rate for ELLs 


21 The ELLs included in the district rates are also included in the overall State rates. 
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Figure 17. ELLs’ In-School and Out-of-School Suspension Rates, by Selected Districts, 
With State ELL In- and Out-of-School Suspension Rates and Overall State In- and Out-of- 
School Suspension Rates Overlaid (2013-14) 



Note: An asterisk indicates that no students were suspended. 
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RQld. District Guidelines for Making Reclassification Decisions 


Key finding 16—Reclassification procedures and decision making: Eight of the 10 sampled districts relied 
most heavily on English language proficiency and content-area assessment data to reclassify students. 


Massachusetts requires districts to use ACCESS for ELLs’ data when making reclassification 
decisions but also recommends that school-based teams consider a range of other relevant data, 
including scores on other standardized and locally administered assessments, students’ grades, 
and student performance as documented by classroom teachers (ESE, 2013). Table 7 displays 
information the sampled districts report using to make reclassification decisions. Districts were 
asked to describe which piece of student information they relied on most heavily to make 
reclassification decisions. Interviewees were not provided with a set list of options to select 
from; rather, this question was open-ended. Selections with asterisks indicate the information the 
districts rely on most heavily for their decisions. 

Eight of the districts rely most heavily on standardized test scores (ACCESS for ELLs and MCAS). 
However, most districts also use other information, including student work or writing samples, 
teacher input, and district benchmark assessments; only Wachusett reports relying on standardized 
assessment scores alone. In two of the districts, Holyoke and Lawrence, district ELL administrators 
did not report relying most heavily on standardized assessment scores for reclassification decisions. 
The finding that reclassification criteria varied across the 10 sampled districts is consistent with 
national studies which show that specific criteria used to reclassify students can vary across states 
and districts within the same state (see Ragan & Lesaux, 2006, for a review). 


Table 7. Information Districts Report Using to Make ELL Reclassification Decisions 


District 

ACCESS 
for ELLs’ 
Scores 

Student 
Work and 
Writing 
Samples 

MCAS 

Scores 

PARCC 

Scores 3 

Teacher 
or Team 
Input 

Benchmark 
and Other 
District 

Assessments 

Other 

Worcester 

• * 


• 





Lawrence 

• 

• 

• 


• * 

• 


Boston 

• * 

• 

• 

• 

• 



Holyoke 

• 

• * 

• 


• 



Lowell 

• * 

• 

#* 


• 



Brockton 

• * 

• 

#* 




Comparison to 
peer performance 

Weymouth 

• * 

• 

• 


• 



Wachusett 

• * 


#* 





Quincy 

• * 

• 

• 


• 



Fall River 

• * 


• 


• 

• 



Note. The * indicates the information the districts reported relying on most heavily to make reclassification 
decisions (no set list of options was provided; this was asked as an open-ended question). 

“Partnership for Assessment of Readiness for College and Careers. In spring 2015, 54 percent of districts in 
Massachusetts will administer PARCC tests to students in Grades 3-8. PARCC is administered in English language 
arts/literacy and mathematics. 
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Phase 1 Conclusion 


This section has examined demographic and descriptive outcomes data for 10 purposively 
selected Massachusetts districts, five of which have the top live highest concentrations of ELL 
students. The large majority of students in these districts are from the United States, and in most 
districts, the most common first language is Spanish. In all 10 districts, the top home language is 
spoken by a large majority of the students, indicating that most of these students could feasibly 
be provided instruction grouped by their first language, with first language support, through dual 
language instruction programs. However, although dual language instruction is allowed in 
Massachusetts, the findings indicate that it is rare in the 10 sample districts; dual language 
programs were reported in only three districts (Boston, Brockton, and Holyoke), and in these 
districts dual language programs make up less than 3 percent of overall ELL programs. 

We also found that in Massachusetts overall, ELLs attend schools with higher concentrations of 
students from low-income households and minority students than do non-ELLs. ELLs tend to 
have lower MCAS scores (with the exception of LELLs, in some cases), lower stability and 
graduation rates, and higher discipline, dropout, and absenteeism rates than the overall student 
populations. However, there are some districts in which these trends do not hold for ELLs or that 
show better rates than the State ELL averages for some or many of these factors. As we note in 
the Discussion section, these districts (Quincy, Wachusett, Lowell, and Brockton) merit further 
study to understand these positive trends. 
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Phase 2 Findings 

The sample for Phase 2 of the study includes kindergarten students attending Massachusetts 
public school districts 22 in the fall of 2003 who were classified as ELLs at school entry. The 
passage of Question 2 legislation in fall 2002 closed the majority of bilingual programs in 
Massachusetts and moved students into sheltered English immersion (SEI) programs. Because 
the new language policy took effect in classrooms in fall 2003, the present study’s cohort 
represents the first kindergarten cohort of ELLs to progress through Massachusetts schools since 
the law went into effect. This cohort-based sample allows the study to follow students through 
virtually their full K-12 careers, providing they remain in Massachusetts. The data set is 
organized in a longitudinal “person-period format” (Singer & Willett, 2003). Students contribute 
one row of data for each year that they attend public school in Massachusetts. 

The kindergarten analytic sample for the time-to-reclassification analysis contained 4,997 ELL 
students at baseline after excluding students not enrolled in a traditional public school district. 
Table 8 shows the demographic characteristics of the kindergarten sample compared to the 
demographic characteristics of non-ELL kindergarteners in Massachusetts in the fall of 2003. 
The majority of ELL students in the initial cohort are from low-income households (70.1 
percent), are Hispanic (52.8 percent), and are U.S.-born (83.4 percent). 24 In comparison, only 
21.5 percent of non-ELL kindergartners in the same year are from low-income households, 8.9 
percent are Hispanic, and 99 percent are U.S.-born. 


22 In the present study, we define “district” as the traditional, multigrade, academically focused school district (e.g., 
Hehr, Grindal, & Eidelman, 2012). We exclude students from our sample who were enrolled in nontraditional public 
school districts at baseline because they represent substantively different environments for ELLs compared to public 
school districts. These included students enrolled in charter schools (n = 63 students) as well as vocational districts, 
separate special education public and private schools, and collaborative schools (although no students were enrolled 
in these types of districts at baseline). After baseline, we excluded 768 students who moved into the above- 
mentioned nontraditional school districts. 

23 Note that RQ2a-l and RQ2a-2 each has its own analytic sample. To simplify reporting, we include the sample 
associated with the reclassification analysis. The analytic sample for the time-to-proficiency analysis includes ELL 
students who remained ELL as of third grade (n = 2,738). 

24 The data do not indicate how many of the U.S.-born students were born in Spanish-speaking U.S. territories such 
as Puerto Rico. 
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Table 8. Characteristics of the Fall 2003 Massachusetts Kindergarten Cohort, by ELL 
Status at Kindergarten Enrollment (2003-04) 



ELL Students (n = 4,997) 

Non-ELL Studentsfii = 63,556) 

Low Income 

67.6% (n = 3,378) 

21.5% («= 13,655) 

Special Education 

7.3% (n = 364) 

9.7% (« = 6,130) 

Female 

48.0% (« = 2,400) 

47.9% ( n = 30,423) 

Asian 

20.4% («= 1,021) 

3.9% (« = 2,473) 

Black 

7.2% (n = 359) 

7.7% (n = 4,895) 

Flispanic 

57.2% (« = 2,860) 

8.9% (n = 5,628) 

White 

15.0% (« = 750) 

79.1% (« = 50,191) 

U.S. Born 

82.4% (« = 4,116) 

99.0% ( n = 62,834) 

Most common first language 

Spanish (56.8%; n = 2,839) 

English (94.3%; n = 59,884) 

Portuguese (9.0%; n = 451) 

Spanish (2.7%; n = 1,711) 

Vietnamese (5.1%; n = 253) 


Chinese (4.4%; « = 221) 


Khmer (2.9%; n = 144) 



The following sections describe the Phase 2 research findings for this cohort of students. 


RQ2a. Average Time to First Reaching Key Academic Milestones for the 2003 
Kindergarten ELL Cohort 


Key finding 17—English language proficiency: The average ELL in our sample first demonstrated English 
proficiency (as measured by the statewide language proficiency assessment [MEPA or ACCESS]) 3.3 years after 
kindergarten entry. 

Key finding 18—Reclassification: The average ELL in the 2003 kindergarten cohort was first reclassified as 
English proficient 2.7 years after kindergarten entry. 

Key finding 19—Reclassification procedures and decision making: Reclassification rates for proficient 
students ranged from 24.4 percent to 51.5 percent, indicating that large numbers of ELLs are not reclassified the 
year following a proficient score on the statewide English language proficiency assessment (MEPA or ACCESS). 


This research question examines the average time it took for the cohort of ELL students who 
entered Massachusetts public schools as kindergartners in the 2003-04 school year to reach two 
academic milestones key for ELLs’ school success. Specifically, this study estimates the time it 
took ELL students to reach proficiency on the statewide English language proficiency 
assessment (RQ2a-l) and the time it takes to be reclassified as fonner ELL (RQ2a-2). 
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RQ2a-l. Time to English Language Proficiency (MEPA and ACCESS) 

To answer RQ2a-l, examining the time to reaching proficiency on the statewide language 
proficiency assessment, we explored a series of discrete-time hazard models to detennine the 
best fitting model” (see Table B6 in Appendix B for a taxonomy of fitted hazard models). We 
included a two-level model with random effects for district to account for the clustering of ELL 
students in Massachusetts districts. Note that we measured time to proficiency for the subset of 
students in the 2003 ELL cohort who remained classified as ELLs until the beginning of third 
grade when the English language proficiency assessment was first administered. 

The median time for a student in our sample to reach English language proficiency (as demonstrated 
on the statewide language proficiency assessment) in Massachusetts is 3.3 years (see Figure 18 and 
Table B6 in Appendix B), several years longer than the median time-to-reclassification (2.7 years). 
However, the actual median may be lower than our data suggest due to the lack of statewide 
proficiency testing for students in Grades K-2 during the time period of our study. 

As shown in detail in Table B6 in Appendix B, student characteristics that had a statistically 
significant, negative association 26 with time to proficiency include receiving special education 
services, being Spanish-speaking, and being from a low-income household. This means that 
students with these characteristics take longer than average to reach English language 
proficiency (as measured by the statewide language proficiency assessment). Being female is 
statistically significant and positively associated with time to proficiency, indicating that girls in 
our sample demonstrated English proficiency, on average, sooner than boys. 


25 We conducted a series of General Linear Hypothesis (GLH) tests to conclude that the best fitting model to this 
data was the model with a quadratic specification of time which also includes a set of covariates—indicators for 
female, low-income status, special education status, race (Black, Asian, and Other with Hispanic as the reference 
group), U.S.-born, and Spanish-speaking (see Table B5 in Appendix B, Model J). 

26 Statistically significant at the p < .05 level. 
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Figure 18. Estimated Hazard and Survivor Functions for Time-to-English Language 
Proficiency for the 2,566 ELLs in the 2003 Kindergarten Cohort, by Years in School, With 
Median Lifetime Overlaid on Plot of Survivor Function 

A. Hazard Functions 



0123456789 10 

Years in School Since Kindergarten 
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Note: Students were not assessed for English proficiency in years prior to 2007. 

RQ2a-2. Time to Reclassification 

To answer the second part of RQ2a, examining the time to first being reclassified from ELL 
status to fonner ELL, we explored a series of discrete-time hazard models to determine the best 
fitting model (see Table B6 in Appendix B for a taxonomy of fitted hazard models for time to 

reclassification). We employed a two-level model with random effects for districts to account for 

28 

the clustering of ELL students within districts. 

The average ELL was reclassified 2.7 years after kindergarten entry. We highlight this point in 
time when half of the sample was exited from ELL instructional services, the “median lifetime,” 
in Panel B of Ligure 19. In Panel A in Ligure 19, the point at which the graph peaks—after seven 
years in Massachusetts schools—indicates when students are at greatest “risk” of being 
reclassified. However, this finding should be interpreted cautiously because as more students in 
the sample are reclassified, the risk set —or the pool of students eligible to be reclassified— 
becomes smaller and the risk of reclassification can appear greater even if the number of kids 
being reclassified is smaller. 

70 

Student characteristics that had a statistically significant negative association" with time to 
reclassification include low-income status. Black, Other Race, and female had a statistically 
significant positive association with reclassification. A negative, statistically significant 
association indicates that these students take longer than average to be reclassified, while a 
positive and statistically significant association indicates groups that exit ELL instructional 
programs faster than average. These results are displayed in Table B6 in the Appendix. 


27 We conducted a series of General Linear Hypothesis (GLH) tests to conclude that the best fitting model to these 
data was the model with a cubic specification of time which also includes a set of covariates—indicators for female, 
low-income status, special education status, race (Black, Asian, and Other, with Hispanic as the reference group), 
U.S.-born and Spanish-speaking (see Table B5 in the Appendix, Model E). 

28 If students were associated with more than one district, we counted the student’s district as that where the student 
had spent the greatest number of years during the period of analysis. 

29 Statistically significant at the p < . 05 level. 
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Figure 19. Estimated Hazard and Survivor Functions for Time-to-Reclassification for the 
4,997 ELLs in the 2003 Kindergarten Cohort, by Years in School, With Median Lifetime 
Overlaid on Plot of Survivor Function 


A. Hazard Functions 



B. Survivor Functions 
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Longitudinal Portrait of the 2003 Kindergarten ELL Cohort 

In 2003, 5,060 students were initially identified as ELL in kindergarten. Of those students, 2,566 
students were included in our sample for time-to-proficiency because they had remained in the 
sample until Grade 3 and consequently had language proficiency data. The 4,997 students 
included in our time-to-reclassification sample represent all the ELL students in kindergarten in 
2003 who were enrolled in public school districts. 

By the final year of the study period (2013-14), 2,787 students of the 2003 kindergarten cohort 
remained in Massachusetts schools. 30 Of these students, 2,491 students (89.4 percent) had been 
reclassified or exited from ELL instructional programs. Of the 296 students who remained 
classified as ELL in 2013-14, more than half (59.1 percent) were also identified as receiving 
special-education services. Note that 7.3 percent (n = 364) of our analytic sample received 
special education services at baseline. The majority of students among this group who remained 
classified as ELL throughout the entire period of the study and were also receiving special 
education services were recorded in the state data set as having “specific learning disabilities” as 
the nature of primary disability (Table 9) and demonstrated a “high” level of need (Table 10 
below). 

Table 9. Nature of Primary Disability for Students Who Remained Classified as ELL 
During the 10-Year Period of Analysis (n = 175; based on 2013-14 data) 


Nature of Primary Disability 

N 

Percent 

Intellectual 

36 

20.6 

Sensory/Hearing 

* 

* 

Communication 

23 

13.1 

Emotional 

17 

9.7 

Health 

20 

11.4 

Specific Learning Disabilities 

68 

38.9 

Multiple Disabilities 

* 

* 

Autism 

* 

* 

Neurological 

* 

* 

Total 

175 

100 


Note. Asterisks represent cell sizes that were too small to report. 


30 This includes all the students from the initial sample who were enrolled in Massachusetts schools in 2013-14; 
some of these may have left and returned. 
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Table 10. Level of Need for Special Education Services for Students Who Remained 
Classified as ELL During the 10-Year Period of Analysis (« = 175; based on 2013-14 data) 


Level of Need 

N 

Percent 

Low—less than 2 hours per week 

* 

* 

Low—2 hours or more per week 

10 

5.7 

Moderate 

95 

54.3 

High 

68 

38.9 

Total 

175 

100 


Note. Asterisks represent cell sizes that were too small to report. 


With respect to demographic characteristics, the students who remained classified as ELL throughout 
the entire 10 years of the period of study were U.S.-bom (100 percent compared to 83.4 percent at 
baseline), Spanish speakers (75.6 percent compared to 55.5 percent at baseline), and from low- 
income households (92.6 percent compared to 70.1 percent at baseline; see Figure 20). 

Figure 20. Characteristics of Students From the 2003-04 Kindergarten Cohort Who 
Remained Classified as ELLs in 2013-14 (« = 296) 



U.S.Bom Low Income First Language Spanish Special Education 

Services 


■ Baseline Long-term ELL 
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RQ2b. Relationship Between English Language Proficiency Attainment and 
Reclassification 

To better understand the relationship between the attainment of English proficiency as measured 
by the MEPA or ACCESS and reclassification, we conducted two sets of descriptive analyses. 
First, we examined the proportion of the 2003-04 kindergarten ELL cohort who scored at the 
proficient level on MEPA while still classified as ELL (i.e., they have not been reclassified as 
former ELL; Table 9). Second, we calculated the proportion of students who were reclassified as 
non-ELL for the school year immediately following attainment of first-time English language 
proficiency (Figure 21). Together, these statistics provide insight into the degree to which the 
language proficiency assessment may be used to make reclassification decisions. 

We generated the subsample in Table 11 by examining the subset of the time-to-reclassification 
analytic sample who were still classified as ELL at the start of third grade. Then, we examined 
the proportion that attained English proficiency for the first time in the spring of third grade. 
Examining the last row in Table 9, we see that the proportion of students in our sample who 
reach proficiency in a given year drops precipitously between the 2007-08 and 2008-09 school 
years. This drop corresponds with a shift in Massachusetts from four levels of ELL proficiency 
to five. This may indicate that districts began using the language proficiency assessment more 
closely to make reclassification decisions such that those who were proficient were typically 
reclassified and thus fewer proportions of nonreclassified ELLs demonstrated proficient scores 
on the language proficiency assessment. 


Table 11. Number and Percentage of Nonreclassified ELLs Reaching English Language 
Proficiency, by School Year 


School Year 

2006- 

07 

2007- 

OS 

2008- 

09 

2009- 

10 

2010- 

11 

2011- 

12 

2012- 

13 

2013- 

14 

N Reached first 
proficiency 

1,054 

569 

38 

50 

73 

53 

16 

15 

A Not proficient 

785 

563 

825 

564 

364 

246 

196 

121 

N total 

1,839 

1,132 

863 

614 

437 

299 

212 

136 

Percent achieving 
proficiency 

57.3% 

50.3% 

4.4% 

8.1% 

16.7% 

17.7% 

7.6% 

11.0% 


Note. School year 2006-07 was the first year the statewide English proficiency test was administered to the 2003 
kindergarten cohort. English language proficiency means scoring at Level 4 on the 2006-07 and 2007-08 
administrations of MEPA; at Level 5 from the 2008-09 to 2011-12 administrations of MEPA; or at Levels 5 or 6 in 
2012-13 and 2013-14 on the ACCESS for ELLs. 

In Figure 21, we show the percentage of the students who were first reclassified as non-ELL the 
year immediately following their first proficient score on the language proficiency assessment. 
Surprisingly few students were reclassified in the first post-proficiency year. Reclassification 
rates for proficient students ranged from 24.4 percent to 51.5 percent (see Figure 21). Future 
analysis should explore the timing of reclassification decisions—specifically, when students 
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score proficient on the language proficiency assessment, when students are typically reclassified 
and when the change is reflected in the statewide database. 31 

Phase 1 findings indicate that currently, districts do rely heavily on statewide language 
proficiency assessment to make reclassification decisions. Eight out of the 10 districts in our 
purposive sample (with the exception of Lawrence and Holyoke) reported relying most heavily 
on students’ ACCESS scores to make reclassification decisions. Of those eight districts, three 
(Lowell, Brockton, and Wachusett) also reported using MCAS scores as the most important data 
point when detennining whether an ELL should be exited from ELL instructional services. 

Figure 21. Percentage of Students First Reclassified the Year After Reaching Proficiency, 
by School Year (2008-09 to 2013-14) 

60 — 



2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014 

School Year 


31 Massachusetts collects student data on October 1 and at the end of the school year. For this study, we used the 
October 1 data collection only. 
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RQ2c. Classified ELLs Scoring Proficient on MCAS 


Key finding 20—Content-area proficiency after reclassification: Greater proportions of former ELLs scored 
proficient on the ELA and mathematics MCAS compared to ELLs. This result is to be expected given that MCAS 
scores are one of the criteria that districts use to reclassify ELLs. However, by eighth grade, only 68 percent and 
40 percent of former ELLs scored pro ficient on the English language arts and mathematics MCAS assessments, 
respectively, compared to 83 percent and 55 percent of never-ELLs statewide. 


In this section, we calculated the percentages of ELLs and fonner ELLs (students who were 
reclassified during the period of analysis) in our sample that scored proficient or higher on the 
MCAS mathematics and English language arts exams. In Table 10, we show that in every year, 
greater proportions of former ELLs scored proficient on the ELA and mathematics assessments 
compared to ELLs. This result is to be expected given that MCAS performance is part of the 
reclassification criteria per state guidance (i.e., students with low MCAS scores are not likely to 
be candidates for reclassification; ESE, 2015). However, by eighth grade, only 68 percent and 
39 percent of former ELLs scored at the proficient or above level in ELA and mathematics 
respectively, compared to 83 percent and 55 percent of never-ELL peers in that same year and 
grade. The proportion of former ELLs performing at the proficient or above level jumps in 10th 
grade to 87 percent and 74 percent in ELA and mathematics (compared to 92 and 81 percent of 
never-ELL 10th graders in that year). However, students who remain in school through 10th 
grade may, as a group, represent higher performing students. 

The Phase 2 finding that fonner ELLs lag behind their never-ELL peers in ELA and mathematics 
may appear to contradict the Phase 1 finding that PELLs in many districts outperformed their 
never-ELL classmates on MCAS (as measured by the Composite Perfonnance Index). However, 
the two phases represent two different samples of students—Phase 1 former ELLs include those 
students who were exited from ELL programs over the past two years. The Phase 2 fonner ELL 
group includes all students who were exited from ELL programs who were ever identified as 
ELL since kindergarten entry. Also, it should be noted that the comparison groups differ as well. 
Phase 1 never-ELLs are reported at the district level in some of the most struggling districts in 
the state. Never-ELLs in the Phase 2 analysis represent all students statewide who were never 
identified as ELL—likely a higher performing group, on average, than the Phase 1 never-ELLs. 

One trend that remains the same across Phase 1 and Phase 2 analyses is that former ELLs 
outperfonned ELLs on MCAS assessments (see Section RQld of the Phase 1 analysis and Table 
12 below). This is not surprising, given that MCAS performance is one data point that districts 
rely on to make reclassification decisions—students with low MCAS performance are not likely 
to be candidates for exit; therefore, the former ELL group, by design, is higher performing on 
content assessments. It is notable, though, that there are ELLs who do score at the proficient 
level or higher; the percentage in several of the years is 20 percent or higher. 
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Table 12. Percentage of ELLs (E) and FELLs (F) Scoring Proficient or Above on the 
English Language Arts and Mathematics MCAS, by School Year 



English 


Math 


Note. MCAS in English Language Arts and Mathematics are administered in Grades 3-8 and 10. An asterisk 
indicates that the 2003 cohort of kindergartners who progressed on time through the grades would have been ninth 
graders in the 2012-13 school year and thus would not have taken the MCAS. We have omitted percentages for this 
school year because those who were assessed were likely those who were retained in grade and thus not 
representative of the ninth graders. 

RQ2d. ELLs’ English Language Proficiency and Content-Area Performance 


Key finding 21—English language proficiency and content-area proficiency: There is a correlation between 
students’ English language proficiency assessment performance and their performance on the English language 
arts and mathematics MCAS assessments in every year of study, with a stronger relationship reported between the 
ELP assessment and the ELA component of the MCAS. 


RQ2d examines the relationship between ELLs’ performance on the statewide English language 
proficiency assessment (e.g., MEPA and ACCESS) and the mathematics and English language 
arts content-area assessments (e.g., MCAS). We computed the correlation between the scaled 
MEPA/ACCESS scores of students in our sample and their scaled scores on the mathematics and 
language arts MCAS exams to determine the relationship between students’ language and 
academic achievement. 

Students’ English language proficiency assessment scores (i.e., MEPA/ACCESS) have a 
statistically significant relationship with their scores on both the English language arts and 
mathematics MCAS exams in every year of our study that students participated in both 
assessments. As would be expected, the correlation between the English language arts MCAS 
and the MEPA/ACCESS is stronger in each year than the correlation between the mathematics 
MCAS and the MEPA/ACCESS scores (Table 13). 


Table 13. Correlations Between Scaled Language Assessments (MEPA/ACCESS) and 
Scaled MCAS Mathematics and English Language Arts Scores, by School Year 


School Year 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

2011-12 

2012-13 

2013-14 

Mathematics 

MCAS 

* 

0.50 

0.49 

0.54 

0.52 

0.47 

0.37 

0.53 

English 

Language 

Arts MCAS 

* 

0.61 

0.64 

0.68 

0.67 

0.67 

0.54 

0.66 


*We were unable to report correlations for the 2006-07 school year because there were no scaled scores reported for 
third graders on the 2007 MCAS administration. All correlations are significant at the .001 level. 
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RQ2e. On-Time Fifth- and Eighth-Grade Promotion 


Key finding 22 — Promotion: Substantially greater proportions of first-time reclassified ELLs were promoted on 
time compared to students who remained classified as ELL. The pattern was true for fifth-grade promotion (83.3 
percent of reclassified ELLs promoted on time compared to 68.2 percent of ELLs) and eighth grade (79.1 percent 
of reclassified ELLs experienced on-time promotion compared to 61.3 percent of ELLs). 


This section addresses RQ2e, examining the proportion of the 2003-04 kindergarten ELL 
cohort 32 who experienced on-time promotion by fifth grade and eighth grade. On-time promotion 
means that a student has not been retained in any grade. Students who continued to be classified 
as ELL in our sample experienced on-time promotion to fifth and eighth grade at lower rates than 
students who had been reclassified (Figure 22). Although 83.3 percent of reclassified students 
reached fifth grade on time in 2008-09, the same was true for only 68.2 percent of ELLs. The 
rate of on-time promotion to eighth grade was 79.1 percent for reclassified students and 61.0 
percent for ELLs still in the sample in 2011-12 (the eighth-grade year for students in the cohort 
who progressed on time through the grades). 

Figure 22. Proportion of ELLs and Lormer ELLs Experiencing On-Time Lifth- and 
Eighth-Grade Promotion 


100 



2008-09 Grade 5 2011-12 Grade 8 


■ELLs ■FELLs 


32 Note that we do not report statewide rates of promotion because the methodology used to calculate retention does 
not provide equivalent comparisons. 


American Institutes for Research 


Massachusetts English Language Learners’ Profiles and Progress—46 





RQ2f: Average Time to Key Educational Milestones by District 


Key finding 23—District variation in time-to-English proficiency: In all districts in the purposive sample with 
sufficient sample sizes, the median time-to-scoring proficient on the statewide English language proficiency 
assessment was at least three years since kindergarten. 

Key finding 24—District variation in time-to-reclassification: There was greater variation in the median time- 
to-reclassification across districts. In some districts, 50 percent of the sample was reclassified within three years: 
Lawrence (1.9 years), Fall River (2.6 years), and Quincy (2.8 years). In other districts, ELLs remained at least 
five years prior to reclassification: Worcester (5.4 years), Lowell (5.3 years), and Brockton (5.3 years). 


Our final research question, RQ2f, examines the average time to reaching key educational milestones 
(i.e., English proficiency and reclassification) in our 10 Phase 1 districts. The district-specific 
estimates for median time to reaching these two milestones were obtained by taking the estimated 
intercepts (or the overall mean across districts) from our final models (see column s denoted “final 
model” in Tables B5 and B6 for English proficiency and reclassification regression coefficient 
estimates, respectively), and adding the estimated district-specific random component. 

Table 14 provides the time in years at which, for each of the Phase 1 sampled districts, 50 
percent of ELLs were reclassified (i.e., the median lifetime [second column]) and demonstrated 
English language proficiency [third column]). The last row in the table provides the median 
lifetime for the average time it took students in this cohort to reach each of the respective 
educational milestones. 

Table 14, Number of Years at Which 50 Percent of ELLs Were Reclassified, by Selected 
Districts 


District 

Median Time to Reclassification 
(years) 

Median Time to English Proficiency 
(years since third grade) 

Worcester 

5.4 

3.3 

Lawrence 

1.9 

3.9 

Boston 

4.8 

3.0 

Holyoke 

3.9 

3.7 

Lowell 

5.3 

3.6 

Brockton 

5.3 

3.1 

Weymouth 

* 

* 

Wachusett 

* 

* 

Quincy 

2.8 

3.0 

Fall River 

2.6 

3.6 

State Average 

2.7 

3.3 


In Table 14 we see that in two districts—Lawrence and Fall River—the median time at which 
ELLs are reclassified is lower than the State average of 2.7 years. In the other districts, there is a 
range of times to median reclassification, from 2.8 years in Quincy to 5.4 years in Worcester. 
These findings are not well correlated with the time to median level of ELL English proficiency, 
as shown in Table 14. Examining the third column in the table shows that in all of the sampled 
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districts that had sufficient data to estimate median lifetimes, it took students at least three years 
since kindergarten to account for 50 percent of students to achieve English proficiency. 

Limitations to Interpretation of Time-to-Reclassification and Time-to-English Proficiency 

When interpreting the discrepancy between the median time to reclassification (2.7 years since 
kindergarten) and time-to proficiency (3.3 years since kindergarten), one must exercise caution 
based on the limitations of the data. First, it is important to emphasize that in these two sets of 
analyses presented in Table 14, columns one (time to reclassification) and two (time to English 
proficiency) use very different samples. The first one includes 2003 ELL cohort kindergartners 
who have reclassification data. The second analysis includes only a subset of the 2003 ELL 
cohort: those who remained as ELL until at least the beginning of third grade. Therefore, it is not 
surprising that the average time to reclassification is lower than the time to proficiency because 
students cannot demonstrate proficiency prior to third grade (it was not tested before this time for 
Grades K, 1, and 2). 

Second, there is a sampling bias particularly for the time-to-proficiency statistics—we are able to 
measure English language proficiency beginning only in third grade when half of the sample has 
already exited ELL instructional services and been reclassified as fonner ELL. The exited 
students were presumably the most high performing ELLs in the sample. Consequently, ELLs 
who remain in the median time-to-proficiency sample are those who have the lowest MCAS 
scores. This is particularly salient because as we show in Table 11 of the report, MCAS scores 
are strongly correlated with English proficiency scores. 

Although it is difficult to interpret the data across these two sets of analyses, one can examine 
findings about each outcome separately. For example, it is noteworthy that Lawrence has 
relatively fast reclassification times and is also one of two districts (along with Holyoke) that 
reported not relying most heavily on English language proficiency and content-area assessments 
to make reclassification decisions (see Table 7; RQld). These districts reported relying most 
heavily on teacher input and on work and writing samples, respectively. However, we cannot 
draw any sweeping conclusions given that district administrators were interviewed about 
reclassification practices in one period of time and the time-to-reclassification data are from a 
longitudinal study that spans 10 years of assessment data. 

Phase 2 Conclusion 

Our longitudinal analysis of ELLs found that the median time to reach proficiency, defined in this 
study as achieving the highest possible level on any given year’s language proficiency exam, was 
3.3 years. This figure is slightly higher than the finding for median time to reclassification (2.7 
years). However, for the reasons described earlier, we cannot draw any sweeping conclusions 
about this discrepancy because largely they involve two slightly different samples of students. 

Other important findings about reclassification included the fact that receiving special education 
services, coming from a low-income household, and being Spanish-speaking were all factors 
negatively associated with time to reclassification. In addition, approximately 12 percent of 
students in the 2003-04 kindergarten cohort were not reclassified by the final year of the study 
(2013-14). Of these students, more than 60 percent received special education services. 
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The longitudinal study also examined student achievement. We found that a substantial number 
of ELLs (20 percent or more, depending on the assessment year) scored at the proficient level or 
higher on the MCAS. Results among FELLs are mixed: on one hand, FELLs experienced higher 
rates of on-time promotion to fifth and eighth grade. However, as we noted in RQ2c 
(summarized in Table 10), large proportions of former ELLs are not proficient by eighth grade. 
We benchmarked these findings to the statewide percent proficient for never-ELLs 33 and there 
are growing gaps between former ELLs and never ELLs in ELA and mathematics from fifth to 
eighth grade. Based on our analysis, 55 percent and 46 percent of fifth-grade FELLs were 
proficient and above in ELA and mathematics compared to 66 percent and 56 percent of fifth- 
grade never-ELLs in the respective year. By eighth grade, 68 percent and 40 percent of FELLs 
were proficient in ELA and mathematics compared to 83 percent and 55 percent of never-ELLs 
in the respective year. 

In the next section, we discuss these findings along with the Phase 1 findings and provide 
recommendations for improving ELL outcomes in Massachusetts. 


33 We calculated these statistics in the respective years and grade levels for never ELLs using the following formula: 
Total number of students proficient and above - number of ELL + former ELL students proficient and above)/(Total 
number of students included - Total number of ELL + former ELL students included) based on the statewide data 
available here: http://profiles.doe.mass.edu/state report/mcas.aspx 
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Discussion 


Our findings reveal that Massachusetts faces some important challenges in addressing the needs 
of its ELLs, including gaps in achievement scores; higher grade retention, dropout, chronic 
absenteeism, and suspension rates; and lower stability and four-year graduation rates. Some of 
these challenges may be related to contextual factors; in Phase 1 of the study, we found that 
ELLs attend schools with higher concentrations of students from low-income households than do 
their non-ELL peers. This situation is not unique to Massachusetts; ELL students live 
disproportionately in poverty (Capps et ah, 2005) and consequently often attend schools 
characterized by large proportions of other ELL students, minorities, and students from low- 
income households. Such segregated environments are associated with lower academic 
achievement for all students (e.g., Orfield, 2001; Orfield & Lee, 2006; Rios-Aguilar & Gandara, 
2012 ). 

Paramount to addressing these and other challenges is understanding the current context of ELL 
instruction and outcomes, a task made easier by Massachusetts’ extensive database of student-, 
school-, and district-level data. These data can be used to pinpoint areas that need improvement 
and to identify areas of promise, but the data themselves can also be improved to better 
understand the needs of the ELL population, especially at the district level. This section 
discusses findings from the descriptive Phase 1 of our study and the longitudinal Phase 2 of our 
study and makes recommendations in three key emergent areas: language instructional services 
for ELLs (including ELLs with disabilities), time to proficiency and reclassification, and data 
collection. 

Language Instructional Services 

As discussed in the Introduction, ELLs have a right to equal educational opportunities, and 
educational agencies are tasked specifically with ensuring that language barriers do not impede 
these opportunities (OCR, 2015). It is therefore crucial that the State monitor and support 
language instructional services for ELLs, especially programs and practices that produce good 
academic outcomes. In this section, we discuss key findings and provide recommendations about 
instructional services for ELLs in the areas of dual language programming, staffing, and services 
for ELLs with disabilities. We also identify some districts that have exhibited better than average 
ELL outcomes and recommend a specific course of action to leverage these districts’ policies 
and programming to improve instruction for ELLs in other districts. 

Language Programming 

Massachusetts is one of the nation’s three English-only states, but dual language instruction, 
which promotes bilingualism and biliteracy along with academic achievement (Faulkner-Bond et 
ah, 2012), is sanctioned. Nonetheless, our examination of ELL enrollment in language learning 
programs in the 10 Phase 1 sample districts suggests that this type of language instruction is rare. 
The lack of dual language programming is unfortunate, because a long-standing and growing 
body of research indicates that first-language instruction promotes gains in English achievement 
and that bilingualism is an asset (e.g., Cummins, 1979; Dressier & Kamil, 2006; Francis, Lesaux, 
& August, 2006; Greene, 1997; Rolstad, Mahoney, & Glass, 2005; Willig, 1985). Our first 
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recommendation, therefore, is to promote dual language instruction in order to draw on the home 
language assets of ELLs in their overall instruction. 

Recommendation 1: Develop and provide a model of language acquisition, with a focus on 
supporting dual language and other similarly promising programming. 

In our Phase 1 sample districts, the number and percentage of speakers of the most common non- 
English language are substantially higher than for the other languages, indicating that a critical 
mass of at least one non-English language is likely possible. Dual language instruction 
promotion might include the following actions: 

■ Educate teachers, administrators, and parents about the benefits and legality of dual 
language instruction. 

■ Provide support to districts to develop dual language programs: 

• Develop and provide curricular materials and resources in multiple languages. 

• Develop and provide job-embedded inservice training and preservice training for 
bilingual teachers, perhaps in partnership with Massachusetts colleges and 
universities (see MBESE, 2009). 

■ Monitor districts’ dual language program services to ensure that they are research based; 
ensure districts are evaluating students’ progress in language proficiency and content 
mastery and making adjustments as necessary. 

The State might also consider supporting pilots of other types of language support programs, 
with the goal of expanding such programs if they show promising results. However, in all cases, 
ensure that instructional programming for ELLs does not needlessly segregate these students 
from English-speaking peers (see OCR, 2015). 

Human Resources to Support a Fast-Growing ELL Population 

The Office for Civil Rights (OCR, 2015) is very clear in its guidance to education agencies that 
instructional programs for ELLs must be adequately staffed by teachers who are qualified to 
effectively instruct ELLs. This is a challenging task for those districts where, following a 
nationwide trend, ELL enrollment has grown rapidly over the past decade. Available data 
suggested that in all 10 of the districts selected for our Phase 1 analysis, the reported proportion 
of teachers providing services to ELL students was substantially lower than the ELL rates of 
enrollment. There are some important caveats to this finding. First, current reporting may 
obscure actual services provided to ELLs if those services are not provided by the students’ 
teachers of record. Second, as of July 2016, all Massachusetts core academic teachers of ELLs 
will be required to hold an SEI teacher endorsement, which means that there will be a substantial 
increase in the number of teachers who are trained to provide SEI to ELLs (Haynes & Paulsen, 
2013). Nonetheless, the State should monitor and support ELL access to good instruction by 
qualified staff, our second recommendation. 

Recommendation 2: Obtain more accurate data on the credentials of teachers providing 
instruction to ELLs, and monitor knowledge of ELL instruction by teachers-of-record. 

Several studies have established connections between teacher knowledge and instructional 
effectiveness in the areas of reading and language (Foorman et ah, 2006; Lane et ah, 2009; 
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Spear-Swerling & Brucker, 2004). The State could change its reporting requirements to better 
monitor who provides instructional services to ELLs and to ensure that they have adequate 
knowledge to do so. Such an approach might include the following actions: 

■ Monitor that there are sufficient qualified staff in districts to provide services to ELLs, 
including certified ESL teachers. Current reporting requirements may not make this 
monitoring feasible; consider streamlining the reporting requirements to ensure 
consistency of data across districts. 

■ All Massachusetts core content teachers who provide instruction to ELLs are required to 
participate in SEI training, but there is currently no system in place to ensure that they are 
knowledgeable about ELL instruction. Develop and administer an assessment of teacher 
knowledge (we recommend that the assessment include an observation component, if 
possible). Include results of the assessment in State data about teachers providing 
language services to ELLs. 

Special Education Services for ELLs 

In our Phase 1 analysis of 10 selected districts, we generally observed similar rates of special 
education identification among ELLs and the general population. However, in Holyoke, ELLs 
appeared to be identified for special education services at higher rates than in the general 
population (33.6 percent compared to 25.1 percent), and in Quincy, ELLs appear to be identified 
at lower rates than in the overall population (4.3 percent compared to 15.5 percent). Furthermore, 
our Phase 2 analysis found that statewide, approximately 12 percent of students who began as 
ELLs in kindergarten did not achieve English proficiency by the end of the study period. The 
majority of these students were also identified as needing special education services, while only 
7.3 percent of the initial sample were classified as needing special education services. 

This finding suggests that although many districts appeared to have similar rates of identification 
among ELLs and other students, there may be factors related to students’ special education status 
impacting their achievement of English proficiency. Indeed, it is possible that in districts that 
appear to have similar ELL identification rates as the general population, underrepresentation in 
lower grades is disguised by overrepresentation in higher grades. This possibility should be 
further examined, as the Office for Civil Rights (2015) has made it clear that ELLs with 
disabilities must be identified in a timely manner. Our third recommendation therefore pertains 
to identifying ELLs with disabilities. 

Recommendation 3: Provide resources and support to districts to identify ELLs with 
disabilities in a timely manner and monitor identification rates at all grade levels. 

The English learner toolkit from the Office of Civil Rights (2015) includes five tools for 
identifying and serving ELLs with disabilities. These could be a useful starting point for assisting 
districts with this important endeavor. In addition to supporting districts, we recommend that the 
State monitor their success in this area. For example, the State might do the following: 

■ Collect data pertaining to when ELLs with disabilities are identified as needing special 
education services. Is this identification occurring as early as it does for non-ELLs, or are 
ELLs at a disadvantage for receiving special education services in the early years of 
school? 
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■ Monitor the special education services provided to ELLs to detennine whether those 
services adequately support ELLs’ language acquisition needs. 

Leverage Lessons From Higher Performing Districts 

As noted previously, our findings indicate that ELLs as a group face both academic and 
nonacademic challenges. However, our Phase 1 analysis of 10 selected districts also revealed 
areas of strength. For example, three districts stood out as especially promising in the area of 
academic achievement. Quincy, Wachusett, and Lowell all had higher average ACCESS for ELL 
and MCAS assessment scores among their ELL populations than the average State ELL 
population. These three districts had lower than average ELL grade retention, dropout, and 
chronic absenteeism rates, and Lowell and Quincy had higher than average ELL four-year 
graduation rates. A fourth district, Brockton, showed promising nonacademic indicators, 
including better than average ELL stability and four-year graduation rates and lower than 
average rates of chronic absenteeism. Brockton also had higher than average SGPs among its 
ELLs in both English language arts and mathematics, indicating progress in academic 
achievement. Our fourth recommendation is to use this information to help districts where ELL 
outcomes are less robust. 

Recommendation 4: Conduct case studies in districts that exhibit higher than average ELL 
performance. 

Case studies such as those conducted by the Rennie Center for Education Research and Policy 
(2007) can help inform policies and programming that are beneficial for ELLs. We recommend 
that the State conduct or support case studies in districts with higher than average ELL 
performance, such as the districts identified in this report (Quincy, Wachusett, Lowell, and 
Brockton). It will be important to include districts with a variety of demographic characteristics 
to be able to examine the relationship among context, policies and programming, and student 
outcomes. For example, Quincy has a high rate of students from low-income households and 
relatively high ELL enrollment, as do Lowell and Brockton. Wachusett has a low rate of ELL 
enrollment and a relatively low rate of enrollment of students from low-income households. The 
State might conduct or support additional analyses to identify other districts with high ELL 
achievement that were not sampled for this study. 

We recommend that case studies examine the following factors: 

■ Staff credentials, training, and experience: As noted in an earlier recommendation, it 
would be important to examine who provides services to ELLs over the course of the 
school week (including general education teachers, ESL specialists, and 
paraprofessionals), the amount of time ELLs spend with these staff, and staff 
certifications as well as preservice and inservice training. 

■ Collaboration and coordination among educators serving ELLs, as well as among school 
district departments: Collaboration and coordination among staff providing school-level 
services to ELLs, including ELL specialists, other specialists (e.g., reading, special 
education, counselors), and general education teachers, as well as among district 
departments providing services helps foster ELL success (August & Hakuta, 1998) and 
should be examined as a potential factor in districts with high rates of ELL achievement. 
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■ ESL specialist tasks : It would be important to examine how district and school ESL 
specialists spend their time. For example, how much of their time is spent providing 
direct instruction to ELLs, administering assessments, modeling instruction for other 
teachers, or providing coaching and other professional development to mainstream 
teachers? 

■ Resource allocation : Districts must divide their resources among many competing needs. 
It would be important to examine how resources are allocated to services for ELLs, 
including professional development funds, curriculum and materials, and direct services 
to students. 

We further recommend that the State use the results of case studies to develop and provide 
technical assistance to other districts, leveraging the support of staff from these districts 
whenever possible. 

Time to Proficiency and Reclassification 

The average ELL in the statewide 2003-04 kindergarten cohort was reclassified relatively 
quickly—after 2.7 years since kindergarten entry. However large numbers of fonner ELLs were 
perfonning below the proficient level by eighth grade (only 68 and 39 percent were proficient in 
ELA and mathematics compared to 83 and 55 percent of never-ELLs in that same year and grade 
statewide). On the other hand, substantially greater proportions of reclassified ELLs were 
promoted on time compared to students who remained classified as ELL. The pattern was true 
for fifth-grade promotion (83.3 percent of reclassified ELLs promoted on time compared to 68.2 
percent of ELLs) and eighth grade (79.1 percent of reclassified ELLs experienced on-time 
promotion compared to 61.3 percent of ELLs). However, we found a significant negative 
relationship between time to reclassification and nonacademic student characteristics, including 
being from a low-income household, or being Asian or Black. We were unable to ascertain the 
reason for these relationships as part of this study, but it is an important point for further 
monitoring and study. This is the basis of our recommendation about time to proficiency and 
reclassification. 

Recommendation 5: Conduct additional research about nonacademic factors related to 
reclassification and act on the findings to ensure that all students have equal opportunities to 
exit from ELL services regardless of background. Monitor performance offormer ELLs who 
may face language-related difficulties as academic content becomes increasingly demanding 
in upper grades. 

In examining the relationship between nonacademic student characteristics and reclassification 
decisions, it would be important to first detennine if the relationships discovered in this study are 
clustered in particular schools or districts. 

Fonner ELLs also wanant attention. Our analyses indicate that there is a growing gap in 
performance between never-ELLs and former ELLs in the upper grades. This is particularly 
important given that the majority of ELLs in Massachusetts are exited in the early elementary 
grades. As academic content becomes increasingly demanding, fonner ELLs may need 
additional support to perfonn on-par with never-ELL peers. 
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Data Collection 

Massachusetts collects, organizes, and maintains an impressive amount of district- and student- 
level data. Nonetheless, both phases of this study revealed additional information that might be 
collected. Primary areas identified for further data collection include information about ELLs’ 
state or territory of origin, about the number of years they attended school in the United States, 
and district-level data that follow ELL cohorts. Our final recommendation is therefore related to 
data collection. 

Recommendation 6: Collect and analyze additional data related to ELLs. 

Additional recommended types of data collection include the following: 

■ Data about ELLs’ states or territories of origin in order to identify ELLs bom in Puerto 
Rico. 

■ Data to track the number of years ELL students have attended school in the United States. 
Currently Massachusetts only measures the time ELL students attend school in the State 
(1-2 years; 3-4 years; 5 or more years). 

■ District-level longitudinal data about ELL cohorts. These data can be used as a way to 
continue the Phase 2 analysis presented in this report. In particular, it would be important 
to track students who are not reclassified and provide targeted interventions for students 
who are in danger of becoming long-tenn ELLs. In addition, these data could be useful 
for setting a “reasonable and realistic” timeline for ELLs to meet English language and 
content area proficiency goals (e.g., Cook, Linquanti, Chinen, & Jung, 2012, p. 29). This 
recommendation is particularly important in light of the U.S. Department of Justice’s 
(2015) guidance for meeting legal obligations to ELLs, which requires education 
agencies to evaluate the effectiveness of districts’ ESL programs. 

In summary, there are some important pockets of promise for ELLs in Massachusetts, but the 
State could leverage additional resources in the areas of language instructional services for ELLs 
(including ELLs with disabilities), time to proficiency and reclassification, and data collection to 
help improve ELL academic outcomes. Of particular importance is ensuring that ELLs with 
disabilities and former ELLs who may be struggling academically in upper grades receive the 
services they need and that all students are considered equally in reclassification decisions. Also 
important is ensuring that districts acquire the resources they need, including human resources, 
to provide adequate services to ELLs so that they may engage meaningfully in their education, as 
they are guaranteed by law. 
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Conclusion 


This study has provided descriptive analysis of Massachusetts ELLs’ demographics and 
academic and nonacademic outcomes as well as a longitudinal analysis of language and content 
outcomes for the 2003-04 kindergarten cohort of ELLs (until 2013-14). Findings reveal that 
some districts are outperfonning others on a range of academic and nonacademic factors. Our 
recommendations included further study of these districts, as well as support for dual language 
instruction, and further data collection and analysis. Further study is emphasized because 
although the analyses presented here provide important information about the state of ELL 
instruction and outcomes in Massachusetts, one limitation presented by the reliance on extant 
quantitative data is that these data could not shed any light on the effectiveness of specific 
programs or program characteristics, such as amount or type of ESL instruction. Another study 
limitation arose from the design of the Phase 2 analysis, which followed a single cohort of ELLs 
starting in kindergarten. This analysis allowed us to see the trajectory only of ELLs who began 
their formal schooling in Massachusetts, but ideally, it would be important to see the trajectories 
of ELLs who arrive later as well. 

Future studies might therefore compare kindergarten trajectories to the trajectories of later 
arrivals as well as to never-ELL trajectories. Additional research might also involve collecting 
and analyzing data about specific subgroups of students, including those who outperform or 
underperform their peers, those who were shown not to be receiving ELL services (e.g., in 
Weymouth and Worcester), and those who belong to groups that our analysis showed tend to 
reach reclassification later (e.g., coming from a low-income household, or being Asian, Black, 
and Spanish-speaking). 

Massachusetts ELLs, like their peers nationwide, tend to face additional challenges in education. 
It is our hope that this study and future studies will infonn improved policies and practices that 
improve ELLs’ educational experiences and outcomes. 
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Appendix A. Research Methods Detail 

Selection of Four Prototypical Districts for Phase 1 Research 

As noted in the main body of the text, 10 Massachusetts districts were selected for Phase 1 of the 
research. Five of these districts were those that had the highest concentrations of ELLs: 
Worcester (35.1 percent ELL), Lawrence (29.9 percent), Boston (29.8 percent), Holyoke (28.5 
percent), and Lowell (26.6 percent). The sixth district, Brockton, was selected because of its high 
concentration of ELLs (20 percent; seventh-largest ELL district statewide) and innovative ELL 
programming. The remaining four districts in the Phase 1 analysis were selected on the basis of 
districtwide concentrations of students from low-income households and growth in English 
language proficiency. More specifically, we selected four prototypical districts that met the 
following criteria: 

■ One district with low numbers of students from low-income families and whose ELL 
population experienced lower growth in English language proficiency outcomes 

■ One district with low numbers of students from low-income families and whose ELL 
population experienced higher growth in English language proficiency outcomes 

■ One district with higher numbers of students from low-income families and whose ELL 
population experienced lower growth in English language proficiency outcomes 

■ One district with higher numbers of students from low-income families and whose ELL 
population experienced higher growth in English language proficiency outcomes 

We relied on two key variables to determine the four prototypical districts—the proportion of 
students from low-income households and the median student growth percentile on ACCESS. 
With respect to the proportion of students from low-income households, we extracted this 
variable from ESE profiles for each district statewide and ranked the districts from the highest to 
lowest proportions of students from low-income households. We flagged those that fell into the 
top and bottom quartiles. With respect to growth in English language proficiency, we relied on 
data files provided by ESE that maintain the districtwide indicator of growth in English language 
proficiency—the median student growth percentile on ACCESS (SGPA). We ranked the districts 
from lowest to highest SGPA score and flagged those that fell in the top and bottom quartiles. 

This process of ranking districts by quartiles on the basis of income and SGPA resulted in four 
bins of districts that corresponded to the four prototypical districts: (1) high income, low growth 
[on English language proficiency]; (2) high income, high growth; (3) low income, low growth; 
and (4) low income, high growth. As we display in Table Al, two districts fall into the high- 
income, high-growth bin, 13 districts fall into the high-income, high-growth bin, 19 districts fall 
into the low-income, low-growth bin, and two districts fall into the low-income, high-growth bin. 
In Table Al, we highlight in green the districts where SGPA fell within the high-growth 
benchmark, as defined by ESE. We highlight in red the districts where SGPA fell within the low- 
growth benchmark as defined by ESE. Notice that there are several districts in the third group— 
low income, low growth—that were not defined as low-growth districts by ESE standards but 
were in the bottom quartile of the SGPA distribution. 


American Institutes for Research 


Massachusetts English Language Learners’ Profiles and Progress—A-l 


Table Al. Display of the Four Bins of Potential Prototypical ELL Districts in Massachusetts 

(Red highlighting indicates ESE-defined low growth on ACCESS; green highlighting indicates ESE-defined high growth on ACCESS; gold 
highlighting indicates ESE selections.) 


(1) High Income, Low Growth 

(2) High Income, High Growth 

(3) Low Income, Low Growth 

(4) Low Income, High Growth 

District 

Low 

SGP 

SGP 

ELL 

District 

Low 

SGP 

SGP 

ELL 

District 

Low 

SGP 

SGP 

ELL 

District 

Low 

SGP 

SGP 

ELL 


Inc % 

% 

Incl # 

% 


Inc % 

% 

Incl # 

% 


Inc % 

% 

Incl # 

% 


Inc % 

% 

Incl # 

% 

Marblehead 

11.2 

47.5 

64 

2.9 

Andover 

6.9 

71.0 

78 

1.9 

Amherst 

43.6 

42.0 

109 

14.0 

Quincy 

47.9 

72.00 

842 

14.5 

Wachusett 

10.9 

49.0 

75 

1.4 

Brookline 

10.7 

82.0 

248 

9.0 

Attleboro 

38.8 

39.0 

317 

8.2 

Tisbury 

36.5 

71.50 

24 

12.9 






Chelmsford 

7.5 

72.0 

79 

2.1 

Chelsea 

85.7 

47.0 

631 

18.9 











Lexington 

6.4 

78.0 

209 

5.5 

Fall River 

79.3 

44.5 

558 

8.5 











Newton 

11.0 

72.0 

520 

7.2 

Fitchburg 

76.8 

45.0 

528 

15.2 











Southborough 

3.3 

72.0 

53 

5.1 

Gill-Montague 

55.2 

37.0 

31 

5.6 











Wayland 

6.7 

69.0 

22 

1.1 

Gloucester 

41.5 

43.0 

59 

2.9 











Wellesley 

5.6 

82.0 

58 

1.7 

Greenfield 

61.0 

39.0 

31 

3.5 











Westborough 

9.5 

74.0 

181 

8.3 

Haverhill 

55.2 

48.0 

321 

7.5 











Westford 

4.5 

75.0 

29 

1.1 

Lowell 

74.3 

43.0 

3079 

29.4 











Weston 

4.8 

81.0 

57 

3.3 

Lynn 

80.8 

50.0 

1594 

17.8 











Weymouth 

4.2 

74.5 

138 

3.1 

Medford 

39.0 

47.0 

215 

6.8 











Winchester 

6.1 

71.0 

106 

3.6 

New Bedford 

79.2 

49.0 

312 

5.5 
















Peabody 

35.9 

48.0 

233 

5.6 
















Pittsfield 

59.2 

30.5 

142 

4.0 
















Rockland 

46.0 

49.5 

22 

1.9 
















Somerville 

66.0 

47.0 

541 

16.8 
















Southbridge 

76.3 

33.5 

180 

17.0 
















Springfield 

89.3 

39.0 

2926 

17.0 
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Table A2. Selection of the Final Four Prototypical Districts on the Basis of: (1) Average Percentage of Students From Low- 
Income Families; (2) Average Student Growth Percentile on ACCESS (SGPA); (3) Average Number of ELL Students Included 
in SGPA; (4) Average Percentage of ELL Students; (5) Median Percentage of Students From Low-Income Families; (6) Median 
SGPA; (7) Median Number of Students Included in SGPA; and (8) Median Percentage of ELL Students in the District 


District name Avg. Low Average Average # ELL Average Median Low Median SGPA Median # Median ELL% (8) 

lnc% (1) SGPA (2) Incl SGPA (3) ELL% (4) lnc%(5) (6) Students Incl 

SGPA (7) 

High Income, Low Growth 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Marblehead 

• 



• 

• 



• 

Wachusett 


• 

• 



• 

• 


High Income, High Growth 

6.7% 

75 

137 

4.1 

6.4% 

74 

79 

3.3% 

Andover 









Brookline 









Chelmsford 







• 


Lexington 





• 




Newton 









Southborough 









Way land 

• 








Wellesley 









Westborough 






• 



Westford 


• 







Weston 








• 

Weymouth 



• 






Winchester 




• 





Low Income, Low Growth 

62.3% 

43.2% 

623 

10.8% 

61.1% 

44.5% 

312 

8.2% 

Amherst 


• 







Attleboro 








• 

Chelsea 



• 






Fall River 




• 


• 



Fitchburg 









Gill-Montague 









Gloucester 
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District name Avg. Low Average Average # ELL Average Median Low Median SGPA Median # Median ELL% (8) 

lnc% (1) SGPA (2) Incl SGPA (3) ELL% (4) lnc%(5) (6) Students Incl 

SGPA (7) 

Greenfield 

• 




• 




Haverhill 









Lowell 









Lynn 









Medford 









New Bedford 







• 


Peabody 









Pittsfield 









Rockland 









Somerville 









Southbridge 









Springfield 









Low Income, High Growth 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Quincy 

• 

• 

• 

• 

• 

• 

• 

• 

Tisbury 










Note: Values of indicate the district in each bin that was closest to the average or median value [or the highest value in the case of high-income/low-growth 
and low-income/high-growth bins]. Because “High Income, Low Growth” and “Low Income, High Growth” had only two districts, highest was used as the 
selection criterion instead of the average values of each of these indicators. 
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After defining the districts within each of the four prototypical bins, our next task was to 
detennine which district to select in each bin. The goal was to select a district within each bin 
that was sufficiently representative of all the districts in that bin. To help determine which 
district to select in each bin, we generated average and median values: percentage of students 
from low-income families; (2) the districtwide SGPA; (3) the number of ELL students included 
in the SGPA calculation; and (4) the percentage of ELLs in the district. 

In each column of Table A2, we display the average/median value of the eight indicators 
(median and average of each of the four indicators). To indicate which district has the closest 
average or median value for the given indicator, we add a bullet point by that prototypical bin. 
We highlight the districts whose average/median values are closest to the average/median for the 
bin. Note that, in the high-income, high-growth bin, there is no district whose average/median 
values are closest to the average/median for the bin more than once. 

We presented these data to the Massachusetts Office of English Language Acquisition and 
Academic Achievement, and staff made the following district selections (highlighted gold in 
Table A2): 

■ High income, low growth: Wachusett 

■ High income, high growth: Weymouth 

■ Low income, low growth: Pall River 

■ Low income, high growth: Quincy 

These selections were based on the districts’ general representativeness in terms of ELL 
population, ELL programming, and socioeconomic characteristics; none of these districts display 
any unusual characteristics within their respective categories. 

Phase 1 Interviews: Guiding Questions for District Administrators 

The following are the questions we posed to district ELL administrators regarding their districts’ 
ELL programming and reclassification policies and procedures. 

1. Please describe the services that ELLs receive in your district for language support. How 
does this vary by proficiency level? 

2. What infonnation does your district rely on to make reclassification decisions (e.g., 

MEPA/ACCESS, MCAS, teacher recommendations, local assessments, etc.)? 

3. How are the different pieces of information weighed to make a reclassification decision? 

4. Who makes the reclassification decision? 

5. How common is it for students to be placed back in ELL services after being reclassified as 
English proficient? 

6. How have these reclassification procedures changed since the shift from MEP A to 
ACCESS? 
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Description of Data Used for Phase 2 Research 

Phase 2 of the research drew primarily on student-level data from three sources: (1) demographic 
data and data on ELL programming maintained in the student information management system 
(SIMS); (2) English language proficiency data maintained in the Massachusetts English 
Proficiency Assessment (MEPA) (fall 2004-spring 2012) and Assessing Comprehension and 
Communication in English State-to-State (ACCESS) for ELLs (spring 2013-present); and 
(3) content-area performance data maintained in the Massachusetts Comprehensive Assessment 
System (MCAS). Each is described in greater detail in the following sections. 

SIMS 

SIMS provides student-level data originally collected for federal and State reporting 
requirements that are available for research purposes. Because SIMS provides student 
demographic and programmatic variables dating back to 2002, it serves as a key resource for 
constructing a longitudinal data set. A unique statewide identifier allows researchers to li nk 
yearly data files over time for each student within the SIMS database and across other data 
sources (e.g., ACCESS, MCAS). We began with the SIMS data to construct our student-level 
data set in Phase 2 of the proposed study. Specifically, we relied on the fall SIMS data collection 
to identify the proposed kindergarten ELL cohort by selecting all students whose value for 
“limited English proficient” (LEP) was equal to 1 in the fall of the 2003-04 school year. We 
relied on this same variable to detennine when a student was reclassified as English proficient 
(when LEP = 0) and to detennine whether a student was promoted on time to fifth and eighth 
grade across the period of analysis (by tracking on-time progression through the grades). 

MEPA/ACCESS 

To detennine whether a given ELL has developed the language skills needed to be able to perform 
grade-level class work in English, the No Child Left Behind (NCLB) legislation mandates 
administration of an annual assessment of ELLs’ comprehension, reading, writing, speaking, and 
listening skills (U.S. Department of Education, 2009). In Massachusetts, all ELLs in Grades K-12 
were administered the MEPA between 2005 and 2012. The 2012-13 school year marked the first 
administration of a new English language proficiency assessment, ACCESS for ELLs. 

MCAS 

Every spring, Massachusetts students in Grades 34 3-8 and Grade 10, including most ELLs, 35 are 
administered the MCAS to assess their performance in mathematics, English language arts, and 
science and technology/engineering. MCAS can be linked with SIMS data starting with the 2002 
MCAS test administration. We capitalized on the unique student identifier common to MCAS, 
SIMS, and MEPA/ACCESS that allowed us to link these files over time for individual students 
and across data sources. We relied on annual MCAS scores to examine the proportion of the 
ELL cohort that reached academic proficiency in English language arts and mathematics while 


34 During 2002-05, fifth- and sixth-grade students were not administered the English language arts component of 
MCAS, and third- and seventh-graders were not administered the mathematics component of MCAS (ESE, 2014c). 

35 ELLs who arrive in U.S. schools after March 1 may be exempt from the English language arts portion of MCAS 
assessment (ESE, 2014d). 
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classified as ELL and following reclassification. We also relied on the MCAS to examine the 
li nk between proficiency on the English language proficiency and content-area assessments. 

Phase 2 Analysis Methods 

In Table A3, we describe each of the key outcomes for the Phase 2 analysis, organized by 
research question. As a first step in the Phase 2 analysis, we constructed a longitudinal data set 
for the 2003 kindergarten ELL cohort by linking the demographic SIMS files over time for 11 
years, beginning with the 2003-04 school year. We capitalized on the unique student identifier, 
called the State Assigned Student Identifier (SASID), to link these files over time. We then 
merged in the English language proficiency (MEPA and ACCESS) data and content-area 
assessment (MCAS) data for the respective years, such that each student in the 2003-04 
kindergarten ELL cohort had demographic, English language proficiency, and content-area 
assessment data in one file during that student’s tenure in the Massachusetts public schools. 


Table A3. Phase 2 Research Questions and Corresponding Outcome Variables (With Data 
Sources in Brackets) 


Research Question 

Outcome Variable(s) [Data Source] 

RQ2a. Average time to exit and proficiency 

RQ2b. Proportion of students scoring at the 
proficient level on ACCESS who were 
subsequently reclassified 

■ Reclassification: Was the student exited from 
ELL services in the given year (1 = yes; 

0 = no)? [SIMS] 

■ English language proficiency: Lor a student 
scoring at the proficient level (Level 6) on 
ACCESS (1 = yes; 0 = no) [ACCESS], was 
student subsequently reclassified (1 = yes, 
reclassified; 0 = no, remained ELL)? [SIMS] 

RQ2c. Proportion of the kindergarten ELL cohort 
that scored proficient on statewide mathematics 
and English language arts content-area 
assessments while classified as ELL and following 
exit from ELL services (i.e., reclassification as 
former ELL) 

■ Content-area proficiency (English language 
arts): Did the student score at the proficient 
level on the English language arts portion of 
the MCAS in the given year (1 = yes; 0 = no)? 
[MCAS] 

■ Content-area proficiency (mathematics): 

Did the student score at the proficient level on 
the mathematics portion of the MCAS in the 
given year (1 = yes; 0 = no)? [MCAS] 

RQ2d. Relationship between English language 
proficiency and academic proficiency 

■ What is the correlation between a student’s 
English language proficiency and MCAS 
scores in a given year (continuous)? 

[ACCESS, MCAS] 

RQ2e. Proportion of ELLs promoted (fifth/eighth 
grade) 

■ Promotion: Was the student promoted on time 
in fifth grade (1 = yes; 0 = no)? [SIMS] 

■ Promotion: Was the student promoted on time 
in eighth grade (1 = yes; 0 = no )? [SIMS] 

RQ2f. District-level variation in reaching 
academic milestones 

■ What was the district-level variation in 
outcomes for RQ2a (reclassification)? 
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In Table A4, we provide an overview of the cohort structure for a student who progresses on 
time 36 through the grades beginning in kindergarten (column 2). In columns 3-5, we display the 
respective English language proficiency and English language arts and mathematics content-area 
assessments that the student would be assigned to take in each of the given grades and years. 


Table A4. Cohort Structure and Testing Map 


Measure 

2003- 

04 

2004- 

OS 

2005- 

06 

2006- 

07 

2007- 

OS 

2008- 

09 

2009- 

10 

2010- 

11 

2011- 

12 

2012- 

13 

2013- 

14 

Grade 

K 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

ELP 

MEPA 

MEPA 

MEPA 

MEPA 

MEPA 

MEPA 

MEPA 

MEPA 

MEPA 

ACCESS 

ACCESS 

ELA 

- 

- 

- 

MCAS 

MCAS 

MCAS 

MCAS 

MCAS 

MCAS 

- 

MCAS 

Math 

- 

- 

- 

MCAS 

MCAS 

MCAS 

MCAS 

MCAS 

MCAS 

- 

MCAS 


Note. ELP = English language proficiency. 


We used the proficiency cutoff as defined by the Massachusetts Department of Education for MEPA 
and ACCESS from MEPA to ACCESS between 2011-12 and 2012-13 by using the highest 
proficiency level for each assessment as our proficiency outcome. We used this data set to answer all 
Phase 2 research questions and relied on three types of analyses: (1) discrete time survival analysis 
(RQ2a-l; RQ2e); (2) descriptive analysis (RQ2a-2; RQ2b; RQ2d); and (3) correlational analysis 
(RQ2c). In the following sections, we describe each set of analyses in greater detail. 

Discrete Time Survival Analysis 

To answer the first part of RQ2a and RQ2e, we fit a set of discrete-time hazard models using 
logistic regression analysis in the person-period data set (Singer & Willett, 2003). First, we 
organized student-level longitudinal data into a person-period format (Singer & Willett, 2003). In 
the data set, students contributed one row of data for each occasion of measurement that they 
were present—that is, one for each of the years that they attended Massachusetts public schools. 
The person-period format permitted us to record the values of variables that were either time- 
varying (e.g., ELL status) or time invariant (e.g., home language) for each participant over time. 

To estimate the average time-to-proficiency or time-to-reclassification (RQ2a), we fitted a two- 
level (students nested within district) polynomial discrete-time hazard model (quadratic for time- 
to-proficiency and cubic for time-to-reclassification) for ELL student i enrolled in district j in 
year or time t after entry at kindergarten, under the assumption that the given student did not 
reach the respective educational milestones (proficiency on the ELP assessment or 
reclassification as English proficient) in the prior (/-1 )th year. A two-level model was used to 
account for the correlations that exist among students who attended the same district. The two- 
level polynomial model is specified as follows: 


36 In RQ2e, we examine the proportion of the kindergarten cohort that is promoted in fifth and eighth grade. 

37 Time to reclassification was modeled from time = 0, but time to ELP was measured from time = 3 because the 
ELP assessment was first administered when students in our analytic sample entered third grade. 

38 We also explored a three-level model with year nested within students within districts but found that the outcome 
variance at the student level was not statistically significantly different from zero. This indicates that after 
accounting for time in the model, the clustering of year-to-year observations within the same student is negligible. 
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Student-level model: 

(1) Logit(hazard RECLASS^f-) 


= a 0 j + a 1 YEAR i j t + a 2 YEAR 2 ij t + +a 3 YEAR 3 ij t + a 4 FEMALEij t 


+ a 5 LOWINCOMEij t + a 6 SPEDij t + a 7 ASIANij t + a 8 BLACKij t 
+ a 9 OTHERRACE ijt + a 10 USBORNi Jt + a^SPANISHijf- 

where logit(a) = Zo,g(a/(l — a)) 


District-level model: 
a oj — Too + u o j 

where a 0] = district-specific intercept; 

7oo = grand-mean intercept (or the log-odds of reaching proficiency or being reclassified each 
year for an average student, given that the student did not reach this milestone the prior years) 

u 0 j — random error associated with district j or how each district’s intercept deviates from the 
overall intercept across districts 

The parameters ai .. .an represent change in log-odds of reaching proficiency or being 
reclassified each year associated with a one-unit change in the value of the covariate. From the 
fitted hazard function, we were able to recover the corresponding fitted survivor function and 
estimate the median-time to proficiency or reclassification statewide. 

Time-varying covariates included years in Massachusetts schools. Time invariant, or fixed, 
covariates included indicators for female, low family income, special education services, ethnic 
background (Asian, Black, other race), whether the student was U.S.- or foreign-born and 
whether the student was Spanish speaking. 

To estimate the average time-to-proficiency or time-to-reclassification for the 10 purposively 
selected districts (RQ2f), we estimated the district-specific intercept from the district-level model 
given above. We then calculated the hazard and survival probabilities using this predicted 
intercept and the regression coefficients given in final model column in Tables B6 and B7, 
respectively, for the time-to-proficiency or time-to-reclassification, respectively. 

Descriptive Analysis 

We reported descriptive statistics to answer the second part of RQ2a, RQ2b, and RQ2d. More 
specifically, to answer the second part of RQ2a—examining the proportion of students scoring 
proficient on MEPA (2003-12) and ACCESS (2013-14) who were subsequently reclassified as 
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English proficient—we reported the percentage of students scoring at Level 5 on MEPA or Level 6 
on ACCESS who were reclassified into mainstream classrooms by fall of the following school year. 

To answer RQ2b—examining the proportion of the kindergarten ELL cohort that scored 
proficient on the Massachusetts Comprehensive Assessment System (MCAS)—we reported the 
percentage of students in from 2006-07 (Grade 3) onward who perfonned at the proficient or 
advanced-proficient level of the respective MCAS mathematics and English language arts 
assessments in each of the tested grades displayed in Table A4, benchmarked to percent 
proficient percentages in the general population. To provide ESE with information on the 
content-area performance of ELLs following reclassification, we disaggregated these reports by 
classification status—ELL and former ELL. 

To answer RQ2d—examining the proportion of the kindergarten ELL cohort that experienced 
on-time promotion by fifth and eighth grade—we relied on the grade-level enrollment data 
maintained in the SIMS database. Based on the baseline sample, we examined the grade 
distribution of students for the 2008-09 and 2011-12 school years and calculated the percentage 
of students in the 2003-04 cohort who were at grade level. 

Correlational Analysis 

We reported correlations to answer RQ2c—examining the relationship between ELLs’ 
perfonnance on the statewide English language proficiency assessment (MEPA/ACCESS) and 
the mathematics and English language arts content-area assessments (MCAS). More specifically, 
we correlated each student’s English language proficiency score from the spring of a given year 
(MEPA) or January/February (ACCESS) with that student’s spring MCAS scores in 
mathematics and English language arts. 
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Appendix B. Data Tables 


Table Bl. Percentage of ELLs Scoring Proficient or Higher, or Making Progress on 
ACCESS for ELLs, and Percentage of ELLs Performing at Each Level of ACCESS for 
ELLs, by District and Number of Years in Massachusetts Schools (2013-14) 


District and 


ELLs 

ELLs 

Making 

Progress 







Years in 
Massachusetts 
Schools 

N 

Scoring 
Proficient 
or Higher 

ELL Students Performing at Each Level of ACCESS 





Level 1 

Level 2 

Level 3 

Level 4 

Level 5 

Level 6 

Boston (All) 

14,400 

19% 

56% 

12% 

15% 

28% 

26% 

15% 

5% 

Year 1 and 2 

5,592 

6% 

60% 

25% 

25% 

30% 

13% 

5% 

1% 

Year 3 and 4 

3,965 

25% 

58% 

3% 

9% 

30% 

33% 

18% 

7% 

5 or more years 

4,820 

30% 

52% 

4% 

7% 

23% 

36% 

23% 

7% 

Brockton (All) 

3,425 

16% 

64% 

14% 

16% 

27% 

27% 

14% 

2% 

Year 1 and 2 

1,353 

4% 

62% 

32% 

27% 

28% 

9% 

3% 

1% 

Year 3 and 4 

1,012 

23% 

67% 

1% 

11% 

32% 

32% 

19% 

4% 

5 or more years 

1,060 

26% 

61% 

2% 

6% 

22% 

44% 

24% 

2% 

Fall River (All) 

848 

16% 

54% 

15% 

19% 

29% 

21% 

12% 

4% 

Year 1 and 2 

376 

7% 

60% 

30% 

27% 

27% 

10% 

6% 

1% 

Year 3 and 4 

279 

19% 

51% 

4% 

18% 

33% 

27% 

14% 

5% 

5 or more years 

192 

31% 

53% 

3% 

6% 

28% 

33% 

21% 

10% 

Holyoke (All) 

1,515 

11% 

45% 

15% 

18% 

30% 

27% 

9% 

2% 

Year 1 and 2 

458 

3% 

51% 

37% 

24% 

27% 

10% 

2% 

1% 

Year 3 and 4 

436 

11% 

46% 

5% 

17% 

32% 

34% 

9% 

3% 

Lawrence (All) 

3,887 

10% 

64% 

16% 

21% 

30% 

23% 

8% 

2% 

Year 1 and 2 

2,085 

4% 

62% 

28% 

29% 

28% 

12% 

3% 

0% 

Year 3 and 4 

1,324 

15% 

64% 

4% 

12% 

35% 

34% 

12% 

3% 

5 or more years 

478 

24% 

67% 

3% 

12% 

24% 

38% 

19% 

4% 

Lowell (All) 

4,083 

21% 

54% 

8% 

11% 

30% 

30% 

14% 

6% 

Year 1 and 2 

1,236 

5% 

56% 

24% 

23% 

35% 

13% 

4% 

1% 

Year 3 and 4 

1,022 

20% 

53% 

2% 

8% 

37% 

34% 

13% 

6% 

5 or more years 

1,824 

32% 

55% 

1% 

4% 

24% 

40% 

22% 

10% 

Quincy (All) 

1,369 

25% 

81% 

12% 

12% 

26% 

25% 

15% 

9% 

Year 1 and 2 

809 

11% 

80% 

20% 

17% 

32% 

20% 

8% 

3% 

Year 3 and 4 

401 

42% 

82% 

1% 

4% 

21% 

33% 

25% 

16% 

5 or more years 

159 

53% 

78% 

1% 

3% 

11% 

32% 

28% 

26% 

Wachusett (All) 

112 

21% 

67% 

8% 

8% 

21% 

41% 

13% 

8% 

Year 1 and 2 

42 

2% 

59% 

19% 

19% 

38% 

21% 

2% 

0% 

Year 3 and 4 

35 

20% 

65% 

0% 

0% 

17% 

63% 

11% 

9% 

5 or more years 

35 

46% 

74% 

3% 

3% 

6% 

43% 

29% 

17% 

Weymouth (All) 

235 

31% 

87% 

14% 

12% 

18% 

26% 

22% 

9% 

Year 1 and 2 

144 

20% 

85% 

22% 

19% 

20% 

18% 

13% 

7% 

Year 3 and 4 

72 

49% 

94% 

0% 

0% 

15% 

36% 

36% 

13% 

5 or more years 

19 

42% 

67% 

0% 

0% 

11% 

47% 

37% 

5% 

Worcester (All) 

7,403 

25% 

63% 

12% 

12% 

25% 

26% 

18% 

7% 

Year 1 and 2 

2,653 

8% 

68% 

29% 

23% 

28% 

13% 

5% 

2% 

Year 3 and 4 

2,116 

29% 

63% 

2% 

8% 

29% 

31% 

20% 

9% 

5 or more years 

2,633 

40% 

61% 

3% 

5% 

18% 

34% 

30% 

10% 
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Table B2. Percentage of ELLs Scoring Proficient or Higher on the MCAS, by District, 
ACCESS for ELLs Assessment Level, and Subject Area (2013-14) 


District and ACCESS 
Level 

Percentage of ELLs Scoring Proficient or Higher 

MCAS 

English Language Arts 

MCAS Mathematics 

MCAS Science 

Boston Overall 

22.8% 

30.2% 

5.7% 

Level 1 

6.2% 

10.3% 

0.0% 

Level 2 

3.3% 

10.6% 

2.3% 

Level 3 

7.2% 

13.1% 

1.6% 

Level 4 

18.8% 

27.2% 

3.7% 

Level 5 

35.0% 

43.8% 

8.6% 

Level 6 

70.9% 

76.0% 

32.0% 

Brockton Overall 

13.6% 

15.8% 

6.4% 

Level 1 

5.3% 

5.3% 

0.0% 

Level 2 

0.0% 

0.9% 

0.0% 

Level 3 

2.4% 

3.7% 

0.7% 

Level 4 

13.5% 

12.8% 

6.3% 

Level 5 

23.5% 

31.5% 

12.1% 

Level 6 

55.8% 

63.5% 

31.6% 

Fall River Overall 

17.1% 

20.5% 

8.3% 

Level 1 

0.0% 

0.0% 

0.0% 

Level 2 

0.0% 

0.0% 

0.0% 

Level 3 

3.4% 

2.6% 

6.1% 

Level 4 

15.9% 

21.7% 

4.1% 

Level 5 

29.9% 

38.5% 

12.0% 

Level 6 

63.3% 

66.7% 

0.0% 

Holyoke Overall 

8.7% 

10.0% 

1.2% 

Level 1 

0.0% 

0.0% 

0.0% 

Level 2 

0.0% 

0.0% 

0.0% 

Level 3 

0.5% 

0.9% 

0.0% 

Level 4 

10.0% 

10.5% 

0.0% 

Level 5 

22.9% 

28.8% 

6.3% 

Level 6 

34.8% 

40.9% 

0.0% 

Lawrence Overall 

14.6% 

24.2% 

4.1% 

Level 1 

0.0% 

0.0% 

0.0% 

Level 2 

0.5% 

1.6% 

0.0% 

Level 3 

3.3% 

10.3% 

0.0% 

Level 4 

17.2% 

29.6% 

5.7% 

Level 5 

33.5% 

52.3% 

14.0% 

Level 6 

63.3% 

78.0% 

0.0% 

Lowell Overall 

0.0% 

0.0% 

0.0% 

Level 1 

26.3% 

30.1% 

5.5% 

Level 2 

0.0% 

0.0% 

0.0% 

Level 3 

1.0% 

2.0% 

0.0% 

Level 4 

2.7% 

5.9% 

1.4% 

Level 5 

22.4% 

26.9% 

2.4% 

Level 6 

43.9% 

48.8% 

5.6% 

Quincy Overall 

35.1% 

57.5% 

25.4% 

Level 1 

0.0% 

0.0% 

0.0% 

Level 2 

0.0% 

15.0% 

0.0% 
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District and ACCESS 
Level 

Percentage of ELLs Scoring Proficient or Higher 

MCAS 

English Language Arts 

MCAS Mathematics 

MCAS Science 

Level 3 

4.6% 

38.8% 

3.3% 

Level 4 

19.1% 

55.2% 

25.0% 

Level 5 

36.2% 

62.9% 

39.3% 

Level 6 

82.4% 

78.9% 

55.6% 

Wachusett Overall 

55.8% 

55.8% 

41.2% 

Level 1 

0.0% 

0.0% 

* 

Level 2 

0.0% 

0.0% 

* 

Level 3 

0.0% 

0.0% 

0.0% 

Level 4 

43.5% 

43.5% 

0.0% 

Level 5 

71.4% 

71.4% 

0.0% 

Level 6 

0.0% 

0.0% 

0.0% 

Weymouth Overall 

21.2% 

25.0% 

0.0% 

Level 1 

0.0% 

0.0% 

0.0% 

Level 2 

0.0% 

0.0% 

* 

Level 3 

0.0% 

0.0% 

0.0% 

Level 4 

10.0% 

15.0% 

0.0% 

Level 5 

21.4% 

17.9% 

0.0% 

Level 6 

0.0% 

0.0% 

* 

Worcester Overall 

27.2% 

21.5% 

8.4% 

Level 1 

5.3% 

2.7% 

0.0% 

Level 2 

0.6% 

1.7% 

0.0% 

Level 3 

4.5% 

5.2% 

0.4% 

Level 4 

19.3% 

12.4% 

3.7% 

Level 5 

41.7% 

31.8% 

10.3% 

Level 6 

68.6% 

62.8% 

45.9% 


American Institutes for Research 


Massachusetts English Language Learners’ Profiles and Progress—B-3 

































Table B3. Median Student Growth Percentile (SGP), by District, Number of Years in 
Massachusetts Schools, and MCAS Subject (2013-14) 


District and Number of Years in Massachusetts Schools 

ELA 

Mathematics 

Boston (All years) 

50 

50 

Year 1 and 2 

47 

65 

Year 3 and 4 

53 

57 

5 or more years 

49 

49 

Brockton (All years) 

58 

54 

Year 1 and 2 

42 

32 

Year 3 and 4 

66 

67 

5 or more years 

57 

51 

Fall River (All years) 

51 

52 

Year 1 and 2 

0 

0 

Year 3 and 4 

52 

55 

5 or more years 

49 

52 

Holyoke (All years) 

36 

40 

Year 1 and 2 

0 

0 

Year 3 and 4 

30 

49 

5 or more years 

37 

37 

Lawrence (All years) 

57 

60 

Year 1 and 2 

53 

63 

Year 3 and 4 

57 

65 

5 or more years 

57 

52 

Lowell (All years) 

49 

49 

Year 1 and 2 

64 

54 

Year 3 and 4 

59 

64 

5 or more years 

48 

47 

Quincy (All years) 

60 

67 

Year 1 and 2 

0 

0 

Year 3 and 4 

57 

80 

5 or more years 

63 

64 

Wachusett (All years) 

49 

69 

Year 1 and 2 

0 

0 

Year 3 and 4 

0 

0 

5 or more years 

54 

69 

Weymouth (All years) 

37 

45 

Year 1 and 2 

0 

0 

Year 3 and 4 

0 

0 

5 or more years 

0 

0 

Worcester (All years) 

55 

50 

Year 1 and 2 

61 

50 

Year 3 and 4 

60 

55 

5 or more years 

53 

49 
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Table B4. MCAS Composite Score Index for ELLs, Former ELLs, and Students Who Were 
Never ELLs in English Language Arts, Mathematics, and Science, by District (2013-14) 


District 

English Language Arts 

Mathematics 

Science 

Boston 

ELL 

59 

59 

43 

Former ELL 

88 

81 

66 

Never ELL 

80 

73 

67 

Brockton 

ELL 

54 

48 

42 

Former ELL 

88 

77 

71 

Never ELL 

81 

71 

69 

Fall River 

ELL 

53 

48 

42 

Former ELL 

68 

56 

54 

Never ELL 

79 

71 

72 

Holyoke 

ELL 

46 

44 

37 

Former ELL 

77 

67 

49 

Never ELL 

72 

65 

61 

Lawrence 

ELL 

50 

54 

39 

Former ELL 

71 

69 

53 

Never ELL 

80 

74 

64 

Lowell 

ELL 

62 

59 

41 

Former ELL 

92 

86 

70 

Never ELL 

81 

76 

70 

Quincy 

ELL 

67 

80 

56 

Former ELL 

85 

85 

76 

Never ELL 

88 

79 

80 

Wachusett 

ELL 

77 

78 

78 

Former ELL 

95 

96 

0 

Never ELL 

94 

90 

90 

Weymouth 

ELL 

55 

59 

37 

Former ELL 

82 

75 

63 

Never ELL 

88 

79 

80 

Worcester 

ELL 

62 

52 

46 

Former ELL 

92 

80 

74 

Never ELL 

83 

71 

73 
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Table B5. Sample Sizes Included in Group Composite Score Index Reports by ELL Status 
and MCAS Subject Area for Each Sampled District and Massachusetts 


District 

ELL Classification 

English 

Language 

Arts 

Mathematics 

Science 

n 

% 

n 

% 

n 

% 

Boston 

English language learner (ELL) 

6,553 

26 

6,611 

26 

2,320 

22 


Former English language learner 
(FELL) 

2,608 

10 

2,613 

10 

1,130 

11 


Students neither ELL nor FELL 

16,299 

64 

16,355 

64 

6,991 

67 

Brockton 

ELL 

1,664 

19 

1,671 

19 

617 

18 


FELL 

558 

6 

559 

6 

238 

7 


Students neither ELL nor FELL 

6,537 

75 

6,539 

75 

2,625 

75 

Fall River 

ELL 

433 

8 

443 

8 

142 

7 


FELL 

81 

1 

82 

1 

35 

2 


Students neither ELL nor FELL 

5,034 

91 

5,037 

91 

1,988 

92 

Holyoke 

ELL 

784 

30 

788 

30 

280 

25 


FELL 

116 

4 

116 

4 

65 

6 


Students neither ELL nor FELL 

1,729 

66 

1,727 

66 

758 

69 

Lawrence 

ELL 

1,429 

21 

1,442 

21 

476 

18 


FELL 

111 

11 

779 

11 

229 

9 


Students neither ELL nor FELL 

4,727 

68 

4,743 

68 

1,970 

74 

Lowell 

ELL 

2,314 

32 

2,320 

32 

793 

28 


FELL 

763 

11 

763 

11 

379 

13 


Students neither ELL nor FELL 

4,086 

57 

4,099 

57 

1,684 

59 

Quincy 

ELL 

396 

8 

410 

9 

127 

6 


FELL 

428 

9 

430 

9 

130 

7 


Students neither ELL nor FELL 

3,864 

82 

3,871 

82 

1,704 

87 

Wachusett 

ELL 

56 

1 

57 

1 

20 

1 


FELL 

25 

1 

25 

1 

3 

0 


Students neither ELL nor FELL 

4,100 

98 

4,104 

98 

1,768 

99 

Weymouth 

ELL 

67 

2 

64 

2 

17 

1 


FELL 

48 

1 

48 

1 

12 

1 


Students neither ELL nor FELL 

3,379 

97 

3,377 

97 

1,438 

98 

Worcester 

ELL 

3,641 

30 

3,656 

30 

1,247 

25 


FELL 

1,051 

9 

1,050 

9 

509 

10 


Students neither ELL nor FELL 

7,405 

61 

7,413 

61 

3,296 

65 

Massachusetts 

ELL 

31,518 

6 

31,834 

6 

10,677 

5 


FELL 

15,959 

3 

16,013 

3 

6,194 

3 


Students neither ELL nor FELL 

441,267 

90 

442,441 

90 

194,569 

92 
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Table B6. Results of Fitting Discrete-Time Hazard Models to the Time-to-First-English 
Language Proficiency for the 2003-04 Kindergarten English Language Learner (ELL) 
Cohort, for the Polynomial Specifications of Time (Standard Error in Parentheses) 


Variable 

Parameter 

Model F 

Model G 

Model H 

Model I 

Model J 



(Linear) 

(Quadratic) 

(Cubic) 

(Quartic) 

(Final) 

INTERCEPT 

Po 

1.105*** 

2.328*** 

0.873 

-27.408*** 

2.060*** 



(0.120) 

(0.250) 

(0.763) 

(3.002) 

(0.252) 

TIME 

Pi 

-0 444*** 

-0.968*** 

-0.088 

22.115*** 

-0.893*** 



(0.018) 

(0.096) 

(0.446) 

(2.355) 

(0.097) 

TIME 2 

P2 


0.048*** 

-0.113 

-6.252*** 

0.043*** 




(0.008) 

(0.080) 

(0.647) 

(0.009) 

TIME 3 

P3 



0.009* 

0.716*** 






(0.004) 

(0.075) 


TIME 4 

P4 




-0.029*** 







(0.003) 


FEMALE 

P5 





0.315*** 







(0.059) 

LOW-INCOME 

P6 





-0.186* 







(0.073) 

SPED 

P7 





-0.723*** 







(0.115) 

ASIAN 

p8 





0.114 







(0.204) 

BLACK 

P 9 





-0.187 







(0.222) 

OTHER RACE 

P10 





-0.040 







(0.207) 

US-BORN 

Pll 





-0.104 







(0.086) 

SPANISH 

Pl2 





-0.559** 







(0.195) 

Between-district 


0.644 

0.612 

0.619 

0.644 

0.508 

variance 


(0.120) 

(0.118) 

(0.118) 

(0.120) 

(0.121) 

Goodness of fit 


7672.191 

7642.037 

7637.9508 

7526.662 

7450.1686 

-2LL 







Deviance-based 



Compare 

Compare 

Compare 

Compare 

hypothesis tests 



MF: H 0 : 

MG: H 0 : 

MH: H 0 : 

MG: H 0 : 




P2=0 

p 3 =0 

P 4 =0 

Ps; P6i P7; Ps; P9; P10; 




r (1)=3.8; 

/ 

x 2 

Plli Pl2 = 0 




P< -05; 

(1)=3.8; 

(1)=3.8; 

X (8)=15.5; 




Reject //(, 

P< -05; 

P < 05; 

P< -05; 





Reject H 0 

Reject //,, 

Reject //„ 


Note, n students = 2,738; n observations = 7,364. M = model; H 0 = null hypothesis. Standard errors in parentheses. * 
p < 0.05, ** p < 0.01, *** p < 0.001. All covariates have been centered at the mean of the baseline value. SPED = 
students received special education services. Hispanic has been excluded as the reference category for race; 
therefore, all interpretations are relative to Hispanic students. 
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Table B7. Results of Fitting Discrete-Time Hazard Models to the Time-to-first- 
Reclassification for the 2003-04 Kindergarten English Language Learner (ELL) Cohort, 
for the Polynomial Specifications of Time (Standard Error in Parentheses) 


Variable 

Parameter 

Model A 

Model B 

Model C 

Model D 

Model E 



(Linear) 

(Quadratic) 

(Cubic) 

(Quartic) 

(Final) 

INTERCEPT 

do 

404 *** 

-1.624*** 

-1.480*** 

-1.483*** 

-1 593 *** 


(0.076) 

(0.083) 

(0.083) 

(0.084) 

(0.081) 

TIME 

Cti 

0.152*** 

0.388*** 

0.050 

0.067 

0.058 



(0.008) 

(0.023) 

(0.047) 

(0.080) 

(0.047) 

TIME 2 

a 2 


-0.029*** 

0.081*** 

0.070 

0.081*** 

TIME 3 



(0.003) 

(0.014) 

(0.041) 

(0.014) 

a 3 



-0.009*** 

-0.007 

-0.009*** 





( 0 . 001 ) 

(0.007) 

( 0 . 001 ) 

TIME 4 

a 4 




0.000 







( 0 . 000 ) 


FEMALE 

a 5 





0.156*** 







(0.039) 

LOW-INCOME 

a 6 





-0.207*** 







(0.047) 

SPED 

a 7 





-0.139 







(0.075) 

ASIAN 

a 8 





0.177 







(0.118) 

BLACK 

d 9 





0.315* 







(0.134) 

OTHER RACE 

ciio 





0.324** 







(0.116) 

US-BORN 

an 





-0.004 







(0.055) 

SPANISH 

dl2 





-0.051 







( 0 . 111 ) 

Between-district 


.603 

0.645 

0.636 

0.636 

0.597 

variance 


(.064) 

(0.069) 

(0.067) 

(0.067) 

(0.065) 

Goodness of fit 
- 2 LL 


18293.06 

18169.23 

18100.22 

18100.14 

17999.75 

Deviance-based 



Compare 

Compare 

Compare 

Compare 

hypothesis tests 



MA: Hi 

MB: H 0 : 

MC: Hi 

MC: Hi 




d 2 =0 

d 3 =0 

d 4 =0 

d 5 ; d 6; d 7 ; d 8; d 9; 




(1)=3.8; 

X 2 (l) = 3-8; 

r(l)=3.8; 

dlO; dn 
=0 




P < -05; 

P< -05; 

P< -05; 




Reject H 0 

Reject //„ 

Reject //„ 

X (8)=15.5; 

P< -05; 

Reject //„ 


Note, n students = 4,997; n observations = 20,846. M = model; H 0 = null hypothesis. Standard errors in parentheses. 
* p < 0.05, ** p < 0.01, *** p < 0.001. All covariates have been centered at the mean of the baseline value. SPED = 
students received special education services. Hispanic has been excluded as the reference category for race; 
therefore, all interpretations are relative to Hispanic students. 
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